Abstract
Existing generative watermarking methods rely on fixed or heuristic entropy thresholds in cross-task generation scenarios, leading to redundant watermark injection, detection noise, and degraded generation quality. To address these limitations, we propose GAMark, a Gaussian Mixture Model (GMM)-based semantics-aware Adaptive entropy threshold waterMarking framework. Motivated by the manifold hypothesis in the logits space, GAMark adopts an offline semantic modeling and online frozen inference paradigm. A GMM is trained offline to characterize multimodal latent semantic states, while during generation, entropy thresholds are dynamically synthesized via Bayesian posterior aggregation over semantic states, enabling automatic avoidance of low-entropy sensitive regions and effective watermark embedding in high-entropy regions. At the detection stage, we further introduce an entropy-weighted statistical test based on state inference, which improves robustness against noise introduced by the inclusion of semantically deterministic tokens. Extensive experiments demonstrate that GAMark consistently improves cross-task generation quality and functional correctness, while maintaining detection accuracy and enhancing watermark robustness and inference efficiency.
Introduction
With the rapid advancement of large language models (LLMs), their applications have extended far beyond conventional open-domain text generation to increasingly complex scenarios, including code generation, logical reasoning, multi-turn dialogue, and cross-domain knowledge integration (Li et al. 2024; Chen et al. 2025; Yang et al. 2026). In such cross-task settings, natural language and formal representations often interleave within a single generation sequence, resulting in pronounced mixed-structure characteristics (Pan et al. 2025; Zhang et al. 2024). However, as the scale of LLM-generated content continues to grow, challenges related to authenticity verification, source attribution, and copyright protection have become increasingly prominent (Chen et al. 2024; Gu and Liu 2026). Consequently, ensuring reliable AI-generated content authentication and accurate content provenance has become a critical requirement for building trustworthy generative AI systems. At the same time, the misuse of generative models may enable the large-scale production of misleading or harmful information, raising serious concerns regarding misinformation prevention and content accountability (He et al. 2024).
In response to the growing concerns of content provenance in large-scale LLM-generated content, text watermarking has emerged as an effective solution by embedding imperceptible statistical signals into generated outputs (Ye et al. 2025). Injecting controlled biases at the logits level constitutes a primary mechanism for watermark embedding. A representative example is the KGW watermark (Kirchenbauer et al. 2023), which partitions the vocabulary into green and red token sets and biases generation toward green tokens for statistical detectability. Subsequent work extends this framework through refined logit biasing, adaptive token partitioning, and entropy-aware detection schemes, exemplified by EWD (Lu et al. 2024), which reweights token contributions to improve detection stability under low-entropy generation. More recently, production-oriented systems such as Google’s SynthID-Text (Dathathri et al. 2024) emphasize lightweight, sampling-time injection and low-latency detection, demonstrating the practical viability of decoding-based watermarking for black-box LLMs.
However, these watermarking methods remain limited in mixed-representation generation (e.g., code, mathematics, and natural language) scenarios. First, at the generation stage, most approaches rely on fixed or heuristic entropy thresholds with limited semantic awareness. This design implicitly assumes stable generation patterns and becomes brittle under frequent semantic mode switches, often perturbing low-entropy deterministic tokens or requiring costly online mechanisms (Zhang et al. 2025). Second, at the detection stage, existing methods treat all tokens uniformly, ignoring generation-time watermark sparsity and allowing low-entropy tokens to dilute detection statistics (Lee et al. 2024).
To address these challenges, we propose GAMark, a Gaussian Mixture Model (GMM)-based adaptive watermarking framework for LLMs. GAMark models latent semantic states via offline representation learning. During watermark generation, it performs frozen online inference with Bayesian-weighted soft state assignments to derive adaptive entropy thresholds, enabling smooth adaptation to semantic mode switches. During detection, GAMark reconstructs semantic states using frozen parameters and applies an entropy-weighted statistical test, effectively suppressing low-entropy noise and enabling robust watermark verification in cross-task scenarios. Figure 1 illustrates a comparison between our method and traditional watermarking approaches. The blue dashed line illustrates a fixed threshold used in traditional watermarking methods, which applies a constant decision boundary regardless of context. The red curve represents the adaptive threshold in our method, which dynamically adjusts according to the inferred semantic state and token entropy. Tokens above the threshold are eligible for watermark injection, while those below are preserved to avoid degrading generation quality.
Contributions. Our contributions are as follows:
-
We propose GAMark, a novel framework that performs offline representation learning and supports frozen online inference for adaptive semantics-aware watermarking in cross-task generation.
-
We design a Bayesian posterior-weighted entropy thresholding strategy for watermark generation, allowing watermark embedding to adapt smoothly across heterogeneous generation regimes.
-
We introduce a state-reconstruction-based entropy-weighted detection method designed to robustly suppress low-entropy background noise.
-
Cross-task evaluations on language generation, code synthesis, and mathematical reasoning demonstrate that GAMark achieves strong generation quality and robust detection performance.
Related work
Logit-Based Watermarking. This paradigm has emerged as a dominant approach for watermarking large language model outputs. A representative example is KGW (Kirchenbauer et al. 2023), which employs key-driven pseudorandom vocabulary partitioning and a fixed bias toward selected tokens to embed detectable statistical signals. Subsequent work refines this paradigm by improving partition stability and detection robustness, including, e.g., UniGram (Zhao et al. 2023) and CTWL (Wang et al. 2023).
Entropy-Conditioned Embedding. To mitigate the instability of pseudorandom partitioning-based watermarking, prior work introduces entropy-aware mechanisms that adapt watermark embedding to token uncertainty. SWEET (Lee et al. 2024) applies strong bias only to high-entropy tokens, preserving the structural integrity of low-entropy content such as code or mathematical expressions. CATMARK (Zhang et al. 2025) further incorporates context-aware thresholding, at the cost of increased computational overhead due to online token classification. Beyond entropy-aware decoding-stage methods, several approaches explore alternative embedding strategies outside the generation process. POSTMARK (Chang et al. 2024) shifts watermark embedding to a post-generation processing stage. RE-MARK-LLM (Zhang et al. 2024) inserts watermark patterns via lightweight text editing without requiring access to the underlying model. In low-entropy scenarios such as code generation and mathematical reasoning, naive watermark injection can easily lead to severe functional errors. To address this critical challenge, adaptive watermarking strategies have been proposed.
Lee et al. (2024) introduced an entropy-threshold-based approach that injects watermarks only at high-uncertainty (high-entropy) positions. Wang et al. 2025 proposed MorphMark, which formulates watermark strength as a multi-objective optimization problem and dynamically adjusts bias parameters according to token-level entropy. More recently, Gu et al. 2025 proposed Invisible Entropy, which leverages feature extractors to accurately localize safe regions, enabling imperceptible watermark embedding in low-entropy tasks.
Entropy-Aware Detection methods leverage token-level uncertainty primarily at the detection stage through post-hoc statistical aggregation, without modifying the watermark generation process. A representative example is EWD (Lu et al. 2024), which reweights token contributions in the detection statistic according to entropy, improving robustness under heterogeneous entropy distributions. This detection line of work has subsequently served as a foundation for follow-up studies that further explore uncertainty-aware detection statistics (Räz 2024; Lee et al. 2024).
Distribution Preservation and Unbiased Watermark Optimization Since fixed-bias watermarking methods such as KGW can easily distort the original token distribution and thereby impair generation quality, recent studies have explored watermarking designs that explicitly preserve the model’s native distributional properties. Hu et al. 2023 first proposed Unbiased Watermarking, which samples from a re-normalized distribution and theoretically ensures that the expected distribution of the watermarked text is consistent with that of the underlying model. Building on this idea, Mao et al. 2025 introduced STA-1 (Sampling One Then Accepting), a two-stage sampling strategy that further mitigates generation risks in low-entropy settings. Chen et al. 2025 proposed MCMARK, which enhances the detection robustness of unbiased watermarking in complex scenarios via a multi-channel vocabulary partitioning strategy.
Research Gap. Existing generative watermarking methods show limited awareness of contextual semantics. Although some distinguish high- and low-entropy regions, they assume stable generation processes and struggle with semantic mode switches in cross-task settings.
Preliminaries
Notation Summary. For clarity, we summarize the main notations used throughout the paper in Table 1.
LLM-generated state definition
The generation process of a large language model can be formulated as an autoregressive probabilistic density estimation problem. Given a context sequence \(x_{ T\), we reject the null hypothesis \(\mathcal {H}_0\) and conclude that the text is watermarked; otherwise, we accept \(\mathcal {H}_0\) and conclude that the text is not watermarked. Algorithm 2 illustrates the watermark detection procedures of the GAMark.
Experiments
We conduct extensive experiments on multiple datasets and models to thoroughly evaluate GAMark in terms of performance, robustness, sensitivity, and inference efficiency. Our experiments are designed to answer the following research questions:
-
1.
Cross-task Performance: How well does GAMark perform across different tasks?
-
2.
Robustness: How robust is GAMark under various conditions?
-
3.
Inference Efficiency: What is the inference efficiency of GAMark?
-
4.
Sensitivity Analysis: How do sensitivity parameters and key components affect the generated content?
Experimental setup
Foundation Models and Datasets. We evaluate the proposed watermarking method using Qwen2.5-7B-Instruct and LLaMA3.1-8B-Instruct models on code generation, mathematical reasoning, and long-context understanding tasks. We evaluate our method on four benchmarks covering diverse task modalities and entropy regimes: C4 (RealNews) (Raffel et al. 2020), representing high-entropy open-domain news and general text generation; HumanEval (Chen 2021), consisting of 164 Python problems for assessing functional correctness under strict syntactic constraints; MBPP (Austin et al. 2021) is a program synthesis benchmark evaluating robustness in concise, low-entropy code generation; and MATH-500 (Hendrycks et al. 2021) features alternating natural language reasoning and formal symbolic derivations to assess adaptability under dynamic entropy transitions.
Baselines. To evaluate the effectiveness of the proposed method, we compare it with five representative watermarking algorithms. For fairness, all methods are evaluated under unified hyperparameter settings, including a greenlist ratio \(\gamma = 0.5\) and a base logit bias \(\delta = 2.0\). The compared baselines are summarized as follows: KGW (Kirchenbauer et al. 2023): One of the earliest and most widely adopted generative watermarking methods. It employs a fixed hashing strategy with a constant logit bias and does not adapt to variations in token entropy across contexts. In our experiments, KGW serves as a reference upper bound for detection performance and a lower bound for generation quality. SWEET (Lee et al. 2024): A static entropy-threshold-based approach that injects watermarks only when the predicted token entropy satisfies \(H(p_t) > \tau\) (e.g., \(\tau = 0.9\)). Tokens with lower entropy remain unchanged. This method represents a simple rule-based adaptive strategy. EWD (Lu et al. 2024): Primarily improves the detection stage through entropy-weighted statistics. In generation, it incorporates an entropy-aware bias scaling mechanism. We include this method as a baseline that considers entropy weighting to evaluate whether simple linear weighting is sufficient for handling complex generation scenarios. SynthID (Dathathri et al. 2024): A watermarking approach based on tournament sampling, which selects tokens from candidate sets through multiple rejection sampling rounds. This method represents an alternative mainstream paradigm that does not rely on logit bias. CATMARK (Zhang et al. 2025): A context-aware method that dynamically adjusts entropy thresholds and requires real-time computation with continuous updates of cluster centers during generation.
Evaluation Metrics and Parameter Settings. We evaluate both generation quality and watermark detectability. For code generation and mathematical reasoning tasks, textual similarity metrics are insufficient. We therefore adopt Pass@k as the primary metric. Given n generated samples per problem and c correct solutions, the unbiased estimator is
In practice, we focus on \(k=1\) (Pass@1), which reflects single-generation reliability. For HumanEval, code is generated via zero-shot prompting and evaluated against reference solutions. For general text generation on C4, we report perplexity (PPL), computed using an oracle model with the same architecture as the generation model to measure linguistic fluency. To assess watermark detectability, we adopt two Z-score-based metrics: AUROC measures the area under the ROC curve and reflects the detector’s threshold-independent discriminative ability. TPR@FPR=5% reports the true positive rate when the false positive rate is constrained to 5%, defined as
The detection threshold is set to \(T=4.0\) (\(\textrm{FPR} < 3.17 \times 10^{-5}\)), and the truncation length is set to \(N=1000\).
Decoding Strategy. During text generation, to balance diversity and quality, we adopt a nucleus sampling (top-p) decoding strategy. Specifically, the sampling temperature is set to \(T = 1.0\), and the cumulative probability threshold is set to \(p = 0.95\), without applying top-k truncation. To ensure reproducibility, all random seeds are fixed throughout the generation process.
GAMark Parameter Settings. For offline semantic modeling, we apply Incremental PCA to project high-dimensional logits into a low-dimensional subspace with dimensionality \(d = 50\). A Gaussian Mixture Model (GMM) with \(N_c = 10\) components is then trained to capture latent semantic states, where full covariance matrices are used to balance model expressiveness and robustness. In the watermark generation stage, the threshold scaling coefficient is set to \(\alpha = 1.0\).
Cross-task performance analysis
Table 2 compares the proposed method with representative baselines across different tasks.
On general text generation tasks (C4), GAMark demonstrates strong imperceptibility, achieving a PPL of 5.96, lower than KGW (7.15) and SWEET (6.65), and outperforming the industrial baseline SynthID (6.10). This advantage arises from semantic state modeling and fine-grained threshold control, preventing unnatural probability perturbations in high-entropy regions and enabling near-lossless text generation.
On syntax-sensitive code generation tasks (HumanEval and MBPP), our method exhibits a clear advantage. Prior watermarking approaches, including KGW, SWEET, and CATMARK, face inherent challenges in preserving functional correctness under low-entropy, structurally constrained generation. In particular, CATMARK must aggressively suppress watermark signals to maintain PASS@1 (74.4%), resulting in poor detectability (AUROC 65.09%, TPR 15.85%). In contrast, our method achieves the highest PASS@1 of 76.2% while substantially improving detection performance, attaining an AUROC of 89.30% and a TPR of 81.70%, more than five times higher than CATMARK. This improvement stems from offline GMM modeling of global syntactic structure, enabling effective watermark injection in low-risk regions without compromising functional correctness.
On the mixed-reasoning task that interleaves natural language and formal symbols (MATH-500), our method demonstrates strong robustness under non-stationary reasoning patterns. On Qwen2.5-7B-Instruct, GAMark attains a PASS@1 of 64.4% while achieving strong detectability (AUROC 97.23%). On LLaMA3.1-8B-Instruct, GAMark maintains very high detection performance (AUROC 99.27%, TPR 97.60%). These results further confirm that Bayesian posterior-weighted soft gating adapts more smoothly to dynamic semantic shifts than hard entropy thresholding.
Robustness analysis
We evaluate the robustness of GAMark on MATH-500 under two highly destructive semantic attacks, namely back-translation and paraphrasing, which represent strong adversarial settings.
Figure 3acompares different methods under back-translation attacks, where text is translated from English to Chinese and back to English using the DeepSeek API. The traditional KGW method (gray curve) degrades sharply, with the dashed curve falling far below the solid one and the AUC dropping from 0.812 to 0.650. In contrast, GAMark (red curve) shows closely aligned solid and dashed curves, maintaining an AUC of 0.910 after the attack, indicating stable semantic representations despite substantial lexical substitutions induced by double translation.
Figure 3bpresents results under a more stringent paraphrasing attack, where text is fully rewritten using the Qwen2.5-7B-Instruct model. Most baseline methods (e.g., SWEET and SynthID) exhibit pronounced bottom-right shifts in their dashed curves, reflecting a substantial increase in false negatives. In contrast, GAMark remains robust, with its AUC decreasing only from 0.972 to 0.880, significantly outperforming the second-best method EWD (0.820). This robustness can be attributed to adaptive entropy thresholding, which focuses watermarking on semantically flexible regions and preserves signals under LLM-based rewriting.
Sensitivity analysis
To analyze the sensitivity of GAMark to key hyperparameters, we conduct a parameter study on the number of clusters K and the threshold scaling coefficient \(\alpha\) on the MATH-500 and HumanEval datasets.
Effect of the Number of GMM Clusters K . The number of clusters K controls the granularity of the semantic state space. Figure 4 shows performance trends on both datasets for \(K \in \{2,4,6,8,10,12,14,16\}\). As K increases from 2 to 10, the detection AUROC on both tasks improves substantially. When \(K < 10\), coarse-grained clustering fails to accurately avoid low-entropy regions, which can disrupt code generation and reasoning logic. At \(K = 10\), Pass@1 reaches its peak on both HumanEval and MATH-500, achieving 76.2% and 64.4%, respectively. However, when K exceeds 10, Pass@1 exhibits a slight decline. Considering both computational complexity and overall performance, we adopt \(K = 10\) as the default setting.
Analysis of the Threshold Scaling Coefficient \(\alpha\). The coefficient \(\alpha\) controls the sensitivity of the adaptive threshold to historical entropy statistics. As shown in Fig. 5, a smaller \(\alpha\) yields stronger detection but degrades generation quality. \(\alpha =0.5\) achieves high TPR (99.2% on HumanEval and 98.5% on MATH-500) at the cost of reduced Pass@1 (56.5% and 52.0%), due to erroneous watermarking of low-entropy tokens. Conversely, a conservative setting \(\alpha =1.5\) restores generation quality close to the no-watermark baseline, but weakens detection with TPR below 40%. We therefore adopt \(\alpha =1.0\), which offers the best trade-off, achieving 76.2% Pass@1 and 81.7% TPR on HumanEval, and 64.4% Pass@1 with 77.4% TPR on MATH-500.
Inference efficiency analysis
Computational efficiency is critical for LLMs watermarking. We evaluate the inference latency and throughput of GAMark and baseline methods on HumanEval, with Table 3 summarizing the overhead during generation and detection.
During generation, GAMark achieves a generation throughput of 35.92 tokens/s, outperforming CATMARK (33.13 tokens/s). This gain mainly stems from our frozen inference design, which shifts computationally intensive manifold modeling to the offline stage. As a result, online inference requires only lightweight matrix projections and GMM posterior estimation. Compared to online clustering methods, GAMark introduces only a small overhead (+3.8%), significantly lower than CATMARK (+9.4%), and even slightly surpasses the static SWEET baseline (35.08 tokens/s). These results demonstrate that the GMM-based computation, after dimensionality reduction, achieves efficiency comparable to entropy-based methods while maintaining adaptive capability.
Compared to CATMARK, which requires real-time updates of cluster centers and repeated high-dimensional distance computations during inference, GAMark shifts the computationally intensive manifold modeling to the offline stage. As a result, online inference only involves lightweight matrix projections and GMM posterior estimation. Empirical results show that, compared to online clustering methods, GAMark introduces only a small additional overhead, which is significantly lower than CATMARK, while even achieving slightly higher throughput than the static SWEET baseline. This demonstrates that, after dimensionality reduction, GMM-based computation achieves efficiency comparable to entropy-based methods while retaining adaptive capability.
Overall, GAMark provides precise semantic awareness without compromising inference speed, demonstrating strong practical viability for real-world deployment.
Ablation study
To verify the effectiveness of the core components in the GAMark framework and quantify their contributions to the overall system performance, we compare the full model with two ablated variants. This comparison is designed to separately evaluate the roles of context-aware GMM clustering and entropy-weighted detection.
w/o GMM Clustering (removing GMM-based clustering): The pretrained GMM module is removed, and semantic states are no longer distinguished. During generation, the adaptive threshold based on clusters is replaced with a global dynamic threshold derived from overall statistics (Global Quantile Thresholding). This variant is designed to evaluate the contribution of semantic state modeling to preserving generation quality, particularly the functionality of code substitution.
w/o Weighted Detection (removing entropy-weighted detection): The GMM-based adaptive injection strategy on the generation side is kept unchanged, while the weighting mechanism in the detection stage is removed (i.e., setting the weight \(w_t = 1\)), reducing the detector to a standard Z-score statistical test. This variant evaluates how the weighting strategy improves the signal-to-noise ratio of detection signals under mixed semantic modes.
Based on the results in Table 4, we draw two conclusions. First, GMM clustering is crucial for functionality preservation. Compared to Full GAMark, removing semantic state modeling ( w/o GMM Clustering) leads to substantial drops in Pass@1 on code-centric tasks (−7.7% on HumanEval and −5.8% on MATH-500). This indicates that a global threshold alone cannot distinguish low-entropy syntactic keywords (e.g., def, return) from high-entropy natural-language comments, causing erroneous watermark injection at critical logic points and resulting in functional failures. Therefore, offline GMM modeling forms the foundation for lossless generation in low-entropy scenarios.
Second, entropy-weighted detection (W-Det.) substantially improves detection SNR. By comparing Full GAMark with the w/o Weighted Detection variant, we observe that removing the entropy-weighting mechanism has little impact on generation quality (Pass@1), as the injection strategy remains unchanged. However, it leads to a substantial decline in detection performance. On HumanEval, the AUROC decreases by 7.15, and the TPR drops from 81.70 This degradation occurs because code and mathematical reasoning tasks often contain a large number of low-entropy tokens that are not watermarked due to the avoidance mechanism. When a standard unweighted Z-score detector is applied, these non-watermarked tokens are treated as background noise, which dilutes the statistical significance of the watermark signal. In contrast, entropy-weighted detection assigns higher weights to high-entropy regions where watermark signals are stronger. This strategy effectively filters background noise and enables more sensitive verification under mixed semantic modes.
Visualized case study
To provide a more intuitive illustration of the effectiveness of the proposed GAMark watermarking method, Figs. 6, 7 apresent side-by-side comparisons of watermark embedding and detection results across different methods on the same task.
The regions labeled ’Ignored’ correspond to prompt tokens. The black bars beneath each token indicate the weights assigned during the Z-score computation. Weight magnitudes are visualized through color intensity, where darker colors correspond to higher weights.
The visualization reveals that GAMark selectively embeds watermark signals in high-entropy tokens while avoiding low-entropy deterministic regions such as code keywords and structural symbols. This selective injection leads to more concentrated detection signals and reduces interference with functional tokens. In contrast, baseline methods such as KGW and SWEET distribute watermark signals more uniformly across tokens, resulting in noisier detection statistics (Fig. 8).
Conclusion
We propose GAMark, a GMM-based adaptive entropy-threshold watermarking framework designed for cross-task generation scenarios in large language models. Motivated by the manifold structure of generation distributions, GAMark introduces a novel paradigm that combines offline semantic modeling with frozen online inference. In the offline stage, incremental generation data and Gaussian Mixture Models are used to construct a multimodal semantic state space, while during generation Bayesian posterior inference dynamically synthesizes optimal entropy thresholds according to semantic state transitions. This design enables intelligent avoidance of low-entropy sensitive regions, such as code keywords, thereby preserving functional correctness while embedding watermark signals in high-entropy regions. Extensive experiments across text generation, code synthesis, and mathematical reasoning demonstrate that GAMark consistently improves detection robustness, attack resistance, and inference efficiency, highlighting its practicality as an effective watermarking solution for complex cross-task generation environments. Overall, GAMark provides a unified and practical solution for adaptive watermarking under heterogeneous semantic conditions, making it well-suited for real-world mixed-generation scenarios.
In future work, we plan to extend this framework to more challenging adversarial settings and further optimize efficiency for large-scale deployment.
Data Availability
No new datasets were generated in this study. All datasets used in this work are publicly available and cited in the reference list.
References
Austin J, Odena A, Nye M, Bosma M, Michalewski H, Dohan D, Jiang E, Cai C, Terry M, Le Q (2021) Program synthesis with large language models. arXiv:2108.07732
Chang Y, Krishna K, Houmansadr A, Wieting JF, Iyyer M (2024) Postmark: A robust blackbox watermark for large language models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. pp 8969–8987
Chen M (2021) Evaluating large language models trained on code. arXiv:2107.03374
Chen H, Zhu T, Liu C, Yu S, Zhou W (2024) High-frequency matters: attack and defense for image-processing model watermarking. IEEE Trans Serv Comput 17(4):1565–1579
Chen H, Zhu T, Zhang L, Liu B, Wang D, Zhou W, Xue M (2025) Queen: Query unlearning against model extraction. IEEE Transactions on Information Forensics and Security
Chen R, Wu Y, Guo J, Huang H (2025) Improved unbiased watermark for large language models. Comput Lang. https://doi.org/10.48550/arXiv.2502.11268
Dathathri S, See A, Ghaisas S, Huang P-S, McAdam R, Welbl J, Bachani V, Kaskasoli A, Stanforth R, Matejovicova T (2024) Scalable watermarking for identifying large language model outputs. Nature 634(8035):818–823
Engels J, Michaud EJ, Liao I, Gurnee W, Tegmark M (2024) Not all language model features are one-dimensionally linear. arXiv:2405.14860
Gu Y, Liu M (2026) Bonc: a framework for automatic cryptanalysis using the implementation code. Cybersecurity 9:121. https://doi.org/10.1186/s42400-026-00556-9
Gu T, Wang Z, Huang K, Yao Y, Zhang X, Yang Y, Chen X (2025) Invisible Entropy: towards safe and efficient low-entropy LLM watermarking. Comput Lang. https://doi.org/10.48550/arXiv.2505.14112
Hendrycks D, Burns C, Kadavath S, Arora A, Basart S, Tang E, Song D, Steinhardt J (2021) Measuring mathematical problem solving with the math dataset. arXiv:2103.03874
He J, Dai H, Sui R (2024) Evilpromptfuzzer: generating inappropriate content based on text-to-image models. Cybersecurity 7:70. https://doi.org/10.1186/s42400-024-00279-9
Hu Z, Chen L, Wu X, Wu Y, Zhang H, Huang H (2023) Unbiased watermark for large language models. Crypt Secur. https://doi.org/10.48550/arXiv.2310.10669
Joshi A, Bhatt D, Modi A (2025) Geometry of decision making in language models. arXiv:2511.20315
Kirchenbauer J, Geiping J, Wen Y, Katz J, Miers I, Goldstein T (2023) A watermark for large language models. International Conference on Machine Learning. pp 17061–17084 (PMLR)
Lee T, Hong S, Ahn J, Hong I, Lee H, Yun S, Shin J, Kim G (2024) Who wrote this code? watermarking for code generation. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4890–4911
Li J, Yang Y, Bai Y, Zhou X, Li Y, Sun H, Liu Y, Si X, Ye Y, Wu Y (2024) Fundamental capabilities of large language models and their applications in domain scenarios: A survey. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11116–11141
Lu Y, Liu A, Yu D, Li J, King I (2024) An entropy-based text watermarking detection method. arXiv:2403.13485
Mao M, Wei D, Chen Z, Fang X, Chau M( 2025) Watermarking large language models: an unbiased and low-risk method. In: Che W, Nabende J, Shutova E, Pilehvar MT (eds) Proceedings of the 63rd annual meeting of the association for computational linguistics, Vol 1: Long Papers. Association for Computational Linguistics, Vienna, Austria, pp 7939–7960
Pan X, Fang J, Wu F, Zhang S, Hu Y-X, Li S, Li X-Y (2025) Guiding large language models in modeling optimization problems via question partitioning. In: Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, pp. 2657–2665
Räz T (2024) Authorship and the politics and ethics of llm watermarks. arXiv:2403.06593
Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(140):1–67
Wang Z, Gu T, Wu B, Yang Y (2025) MorphMark: flexible adaptive watermarking for large language models. Crypt Secur. https://doi.org/10.48550/arXiv.2505.11541
Wang L, Yang W, Chen D, Zhou H, Lin Y, Meng F, Zhou J, Sun X (2023) Towards codable watermarking for injecting multi-bits information to llms. arXiv:2307.15992
Yang X, Zhong R, Chen Y, al (2026) Cti-thinker: an llm-driven system for cti knowledge graph construction and attack reasoning. Cybersecurity 9, 106 https://doi.org/10.1186/s42400-025-00505-y
Ye P, Ren H, Li Z, Yan A, Yan H, Wang S, Li J (2025) Securing large language models: A survey of watermarking and fingerprinting techniques. ACM Computing Surveys
Zhang Y, Chen X, Jin B, Wang S, Ji S, Wang W, Han J (2024) A comprehensive survey of scientific large language models and their applications in scientific discovery. arXiv:2406.10833
Zhang Y, Liu S, Yang X, Hu X (2025) Catmark: A context-aware thresholding framework for robust cross-task watermarking in large language models. arXiv:2510.02342
Zhang R, Hussain SS, Neekhara P, Koushanfar F (2023) \(\{\)REMARK-LLM\(\}\): A robust and efficient watermarking framework for generative large language models. In: 33rd USENIX Security Symposium (USENIX Security 24), pp. 1813–1830 x
Zhao X, Ananth P, Li L, Wang Y-X (2023) Provable robust watermarking for ai-generated text.arXiv:2306.17439
Acknowledgements
Not applicable.
Funding
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant No. 62472122, the National Key Research and Development Program of China (Grant No. 2024YFB31NL00101), the National Natural Science Foundation of China (Grant No. U22A2036), the Fundamental Research Funds for the Central Universities (Grant No. HIT.OCEF.2021007).
Author information
Authors and Affiliations
Contributions
Jing Zhao and Hongwei Yang contributed to conceptualization and methodology. Jing Zhao and Hengji Dong developed and implemented the code, while Hengji Dong conducted model training and validation. The initial manuscript was drafted by Jing Zhao, Hongwei Yang, and Hengji Dong. Hui He and Weizhe Zhang provided supervision, resources, and critical revision of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Zhao, J., Yang, H., Dong, H. et al. Gamark: adaptive entropy-threshold watermarking via gaussian mixture modeling for large language models. Cybersecurity 9, 211 (2026). https://doi.org/10.1186/s42400-026-00607-1
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1186/s42400-026-00607-1
Facts Only
* Jing Zhao, Hongwei Yang, and Hengji Dong developed GAMark.
* The project was supported by the National Natural Science Foundation of China and the National Key Research and Development Program of China.
* GAMark is a watermarking framework for Large Language Models (LLMs) using a Gaussian Mixture Model (GMM).
* The system utilizes an offline representation learning phase and a frozen online inference phase.
* Evaluations were conducted using Qwen2.5-7B-Instruct and LLaMA3.1-8B-Instruct models.
* Test benchmarks included C4 (RealNews), HumanEval, MBPP, and MATH-500.
* The framework was compared against KGW, SWEET, EWD, SynthID, and CATMARK.
* Performance metrics used include Pass@1, Perplexity (PPL), AUROC, and TPR@FPR=5%.
* The system employs Incremental PCA for dimensionality reduction to 50 dimensions.
* The GMM was configured with 10 components and full covariance matrices.
Executive Summary
GAMark addresses the technical tension between watermark detectability and the functional integrity of LLM outputs, particularly in cross-task scenarios like code generation and mathematical reasoning. Traditional watermarking often relies on fixed entropy thresholds, which can corrupt deterministic tokens—such as syntax keywords in Python—leading to functional errors. To resolve this, GAMark implements a GMM-based approach that models latent semantic states offline, allowing the system to dynamically adjust entropy thresholds during generation. This ensures that watermarks are embedded in high-entropy, semantically flexible regions while leaving low-entropy, critical tokens untouched.
The framework demonstrates improved performance across diverse benchmarks, maintaining higher Pass@1 rates in coding tasks and better linguistic fluency in general text compared to several baselines. Additionally, it shows increased resilience against adversarial attacks such as back-translation and paraphrasing. While the system introduces a slight computational overhead, the use of frozen online inference keeps latency low. The results suggest that semantic awareness in the watermarking process significantly reduces the trade-off between content authenticity and generation quality.
Full Take
This research enters the field as a rigorous academic contribution to AI provenance. The methodology is sound, leveraging a standard GMM architecture and Incremental PCA to handle the high-dimensionality of logit spaces. By shifting the heavy lifting to an offline phase, the authors solve a primary bottleneck of prior adaptive methods: the computational cost of real-time clustering. A peer reviewer would likely note that while the AUROC and Pass@1 improvements are statistically significant, the reliance on a fixed number of clusters ($K=10$) suggests a heuristic optimization that may vary across different model architectures or larger vocabulary sets.
The evidence supports the claim that semantic-aware gating preserves functional correctness better than static thresholds. However, the authors' claim of "robustness" against paraphrasing attacks should be viewed with nuance; while GAMark outperforms baselines, an AUC drop from 0.972 to 0.880 indicates that no current logit-based watermark is truly immutable against sophisticated LLM-driven rewriting. The novelty is justified as it moves the field from "entropy-aware" (simple uncertainty) to "semantics-aware" (contextual state).
For this to matter outside the lab, the industry must standardize detection protocols, as the effectiveness of GAMark depends on the detector having access to the frozen GMM parameters used during generation.
Bridge Questions:
1. How does the GMM's performance degrade as the gap between the offline training data and the online generation distribution increases?
2. Could a generative adversarial network (GAN) be trained specifically to identify and strip these "semantically flexible" watermark signals?
Counterstrike Scan: The content is a standard academic presentation of a technical solution. It does not match the pattern of a coordinated influence campaign, as it provides detailed ablation studies, cites competitors fairly, and acknowledges performance trade-offs.