Dataset and Setting

The experiments in this paper were conducted on the nested datasets GENIA [43], ACE2005 [44], and the non-nested dataset CoNLL03 [45]. GENIA is a biomedical dataset collected from 2000 abstracts on PubMed in MEDLINE. Following the method of Li et al. [27], the dataset is split into training, validation, and test sets in a ratio of 8.1:1.1:0.9. ACE2005 is a nested entity dataset that contains seven types of named entities. For the English ACE2005 dataset, the method of Shen et al. [9] was used, while for the Chinese ACE2005 dataset, the method of Chen et al. [35] was used to split it into training, validation, and test sets. CoNLL03 is a news domain dataset based on the Reuters news corpus, containing four types of named entities.

In the main experiments across all datasets, bert-base-cased [46] was consistently used. To ensure a fair comparison, bert-base-chinese [46] was additionally employed for the ACE2005-Chinese dataset, and BioBERT [47]was additionally adopted for the GENIA dataset. In subsequent analysis experiments, bert-base-cased was uniformly used to maintain consistency. The hyperparameters are set as in Table 1.

Table 1 Hyperparameters and valuesValuation Metrics

We evaluate model performance using standard metrics for named entity recognition: precision, recall, and F1-score. Precision measures the proportion of predicted entity spans that are correct, while recall measures the proportion of ground-truth entities that are successfully recovered by the model. The F1-score is the harmonic mean of precision and recall, providing a balanced assessment of overall performance.

Formally, let TP, FP, and FN denote the numbers of true positives, false positives, and false negatives, respectively. Then,

$$\begin \text = \frac \end$$

(21)

$$\begin \text = \frac \end$$

(22)

$$\begin \text = 2 \cdot \frac \cdot \text } + \text } \end$$

(23)

An entity prediction is considered correct only if both its type and its exact span boundaries match those of a ground-truth entity.

Comparison ExperimentEnglish Nested Dataset

First, we conduct experiments on the English nested dataset.To validate the effectiveness of the model, we conducts comparative experiments with the state-of-the-art NER models proposed in recent years. Comparison models are divided into three categories: sequence labeling-based methods, span-based methods, and other methods. Sequence labeling-based methods: Layered [4] uses dynamically stacked planar NER layers to address the nested entity recognition task. Pyramid [5] designs pyramid and anti-pyramid layers to decode nested entities.

Span-based methods: BiaffineNER [28] scores all candidate spans in one shot with a biaffine attention mechanism, jointly decoding entity boundaries and their types. Locate-and-Label [9] performs entity localization through boundary regression and classifies entities. W2NER [27] transforms the NER problem into a unified framework based on word-pair relations. Triaffine [48] proposes a triaffine mechanism to integrate internal labels, boundaries, and tags in NER. Binder [49] reformulates NER as a span-type vector matching problem, using contrastive learning to align correct entity spans with their corresponding types in a shared embedding space. DiffusionNER [31] treats NER as a boundary denoising diffusion process, where noisy fragments are regarded as spans, which are then classified.

Other methods: Seq2Seq [7] formulates NER as a sequence generation task. BartNER [50] transforms NER into a span-level sequence generation problem using BART. PIQN [33] defines learnable global query instances to extract and recognize entities in parallel. PromptNER [34] unifies entity localization and entity type classification by filling position slots and type slots to predict entities. Controlled Attention [35] propose a method based on controlled attention, inserting special symbols on both sides of entities in text to help the model perceive entity boundary information.

All results are taken directly from the original papers.

Table 2 Comparison of experimental results on nested English datasets.All experiments were run five times with different random seeds. The results report the average performance ± standard deviation across runs.The “-” sign indicates that the model omitted the experimental results. Note: Bold values indicate the best performance

The results are shown in Table 2. Our model achieves best F1 scores on both the ACE-2005 and GENIA named entity recognition benchmarks. Specifically, it exhibits consistently high precision but relatively lower recall on both datasets. This behavior can be attributed to the Prototype Vector Penalty Term and the contrastive learning loss incorporated into the objective function, which encourage the model to map true entities into compact semantic clusters in the learned metric space while pushing negative instances toward more dispersed regions. This structure effectively reduces false positives, thereby improving precision at the cost of some recall.

On ACE-2005, the model attains a precision of 88.62%, a recall of 86.53%, and an overall F1 score of 87.73%, demonstrating strong discriminative capability in suppressing erroneous predictions. On GENIA, the model achieves a more balanced trade-off between precision and recall, yielding an F1 score of 81.82. This improvement stems from our attention-based mechanism that jointly models the entity and its contextual semantics. In biomedical texts such as GENIA, many named entities appear as abbreviations whose interpretation heavily depends on surrounding context. Nevertheless, it should be noted that our recall remains slightly inferior to that of certain baselines, primarily because the current approach does not explicitly handle discontinuous entities, which are prevalent in the GENIA dataset.

English Non-Nested Dataset

Then, we conduct experiments on the English non-nested dataset.Nested entities often exhibit ambiguous representations due to shared contextual information. In existing prototype methods, this ambiguity in nested entity representations makes it difficult to accurately compute the distance between prototype and entity representations. To validate the advantage of the sentence-level prototype method proposed in this paper over existing prototype methods, comparative experiments are conducted on the non-nested dataset CoNLL03 [45]. The comparison models are divided into two categories based on different entity representations: Token-level prototype methods and span-level prototype methods.

Token-level prototype methods: ProtoNet [16] defines the NER task as a sequence labeling task and constructs Token-level prototypes. ProtoNet+P&D [18] extends ProtoNet [16] by introducing label dependency transfer, where abstract label dependencies are copied from the source domain to the target domain. StructShot [17] further improves dependency transfer by averaging the abstract dependencies over the target domain.

Span-level prototype methods: EP-Net [14] is the first to propose constructing entity-level prototypes. This class of prototype-based NER methods is no longer limited by label dependencies. It constructs entity-level prototypes, trains them from scratch, spreads the prototypes, and calculates the distance between spans and prototypes in the semantic space to recognize named entities. TaGuProto [15] implements entity-level prototypes that integrate entity classification hierarchical information. This method first obtains type-independent span representations via sequence labeling, then incorporates entity classification hierarchical information into the entity-level prototypes.

Non-prototype methods: BiaffineNER [28] scores all candidate spans in one shot with a biaffine attention mechanism, jointly decoding entity boundaries and their types. BERT-MRC [29] frames NER as a machine-reading-comprehension task: for each entity-type question, BERT extracts the matching answer spans from the text. Chen [35] et al. propose a method based on controlled attention.We apply this method to the CoNLL-2003 dataset. DiffusionNER [31] treats NER as a boundary denoising diffusion process, where noisy fragments are regarded as spans, which are then classified. MDNER [32] also employs the diffusion-denoising idea, progressively restoring precise entity spans from random noise.

Results marked with an asterisk (*) denote experiments reproduced by us using the original authors’ code on the specified datasets; all other results are taken directly from the original papers.

Table 3 Comparison of experimental results on non-nested English datasets.The comparison includes both prototypical and non-prototypical methods.Mean ± standard deviation over five runs. Note: Bold values indicate the best performance

The results are shown in Table 3. Our proposed sentence-level prototypical method achieves a 6.37% improvement in F1 score over TaGuProto.Meanwhile, our approach yields a smaller standard deviation, indicating that our prototypical representation learning is more stable and robust than previous prototype-based methods. In contrast to prior approaches that rely solely on Token-level or span-level prototypes, our method introduces attention cues that explicitly indicate entity boundaries within the sentence. This enables the model to accurately perceive entity boundaries and establish global semantic dependencies between entities and their surrounding sentence context. By incorporating such contextual signals during metric space learning, our approach enriches entity representations beyond the local semantics captured by traditional prototype-based methods.To further analyze the distinction from span-level prototypical methods, we provide visualizations of the proposed attention mechanism in Fig. 3, which illustrate how contextual semantics are integrated into entity representations rather than relying exclusively on span-internal features.

Additionally, we compare our method against several non-prototypical baselines. Although it slightly underperforms the current state-of-the-art MDNER method, it consistently outperforms multiple existing approaches, demonstrating strong overall competitiveness.

Chinese Nested Dataset

Lastly, we conduct experiments on the Chinese nested dataset.To evaluate the performance of the proposed method across different languages, comparative experiments were conducted with methods such as NNBA [23], Locate-and-Label [21], W2NER [27], and Chen [35], among others.

The NNBA [23] model uses boundary information to segment the initial text and employs Long Short-Term Memory (LSTM) networks to capture contextual semantic information of text fragments for entity type prediction. Locate-and-Label and W2NER are span-based methods, while Controlled Attention is belong to other methods, which were used as comparison models in the English corpus mentioned earlier.

Results marked with an asterisk (*) denote experiments reproduced by us using the original authors’ code on the specified datasets; all other results are taken directly from the original papers.

Table 4 Comparison of experimental results on nested Chinese datasets. Note: Bold values indicate the best performance

The results are shown in Table 4, where the model achieves the highest precision, recall, and F1 score. This result demonstrates the effectiveness of the proposed sentence-level prototype method in the Chinese data set..

Ablation Study

To verify the effectiveness of individual components in our model, we conduct ablation studies, which are divided into two parts: ablation experiments related to the loss functions, and ablation experiments concerning the attention cues [S] and [E].

Loss Functions

Regarding the loss function, we remove individual loss terms or their combinations to validate their effectiveness. All experiments across the three datasets consistently employ BERT-base as the encoder. As shown in Table 5, the F1 scores consistently degrade upon removing any component of the loss function. Notably, the performance drop is most pronounced when \(\mathcal _}\) is removed. A significant decline is also observed when both \(\mathcal _}\) and \(\mathcal _}\) are removed simultaneously, whereas ablating either one alone results in relatively minor performance changes.

This behavior can be explained as follows: \(\mathcal _}\) encourages prototype vectors of different entity types to be well-separated, thereby facilitating the formation of clear decision boundaries. Meanwhile, both \(\mathcal _}\) and \(\mathcal _}\) are grounded in contrastive learning principles,\(\mathcal _}\) pulls positive entity instances closer together while pushing negative ones apart, and \(\mathcal _}\) draws entity representations toward their corresponding positive prototypes and away from negative prototypes. Together, these two losses promote the formation of compact and discriminative clusters of specific type.

In summary, all loss components jointly contribute to the overall objective of prototype-based representation learning, and their integration is essential for achieving optimal performance.

Table 5 Ablation study results of the loss functions. The ACE2005, GENIA, and CoNLL2003 datasets all use BERT-base as the encoder, with experimental settings identical to those of the main experiments. We report the impact on performance when removing a specific loss function or combinations thereof across the three datasetsAttention Cues

In the semantic fusion module, we extract the vector representations of the [CLS] token along with the special markers [S] and [E], and concatenate them into a unified feature vector. This concatenated representation is then passed through a multilayer perceptron (MLP) to produce a compact semantic embedding for the entity instance. This design is motivated by the observation that the [CLS] token captures context-aware global semantic information, whereas [S] and [E] encode entity-specific local semantics corresponding to the start and end positions of the target entity within the sentence.

To systematically evaluate the impact of different feature fusion strategies on model performance, we conduct experiments on three widely used benchmark datasets: ACE2005, GENIA, and CoNLL2003. Specifically, we consider four fusion configurations: fusing [CLS] with [S]; fusing [CLS] with [E]; jointly fusing [CLS], [S], and [E]. Furthermore, to assess the effectiveness of the proposed attention cues ([S] and [E]), we introduce an additional baseline,[CLS]+[UNK]+[UNK],where [UNK] denotes BERT’s generic placeholder for out-of-vocabulary tokens.

As shown in Table 6, the third strategy (CLS+[S]+[E]) consistently achieves the highest F1 scores across all three datasets, outperforming the other three variants. This superior performance can be attributed to the complementary nature of the fused signals: the global contextual semantics from [CLS] and the precise boundary-aware local semantics from [S] and [E] jointly provide richer and more discriminative cues for entity classification. These results substantiate the efficacy and rationality of our proposed fusion mechanism.

In contrast, the [CLS]+[UNK]+[UNK] configuration exhibits a notable performance drop. We identify two primary reasons for this degradation. First, although [UNK] tokens are inserted at entity boundaries, they lack distinct semantic roles for start versus end positions, leading to ambiguous boundary representations that impair the model’s ability to accurately localize entities. Second, since [UNK] inherently serves as a generic symbol for any unknown word in BERT, it carries noisy and non-specific semantic content, thereby diluting its capacity to exclusively encode entity boundary information.

To intuitively reveal the role of the attention cues [S] and [E] and to corroborate our theoretical analysis, we select one sentence from the CoNLL2003 dataset and visualize the attention flow under three configurations (see Fig. 3): (a) both [S] and [E] are inserted; (b) only [UNK] tokens are used as placeholders; (c) no extra cues are provided. In Fig. 3(a), [S] and [E] attend to each other and concentrate on the entity, successfully capturing its boundaries and the semantic dependencies with the context. By contrast, in Fig. 3(b) the two [UNK] tokens receive scattered and nearly identical attention, making the boundaries indistinguishable. In Fig. 3(c), without any guiding cues, the attention is almost uniformly distributed across all tokens, failing to focus on the entity span. These observations align well with our analysis.

In summary, the experimental results demonstrate the effectiveness of the proposed attention cues ([S] and [E]) in enhancing entity representation and recognition performance.

Table 6 Performance comparison of different fusion strategies on ACE2005, GENIA, and CoNLL03 datasets. Three datasets all use Bert-base as the encoder. Note: Bold values indicate the best performanceFig. 3The alternative text for this image may have been generated using AI.

Attention flow visualization under three configurations. (a) and (c) are generated from the same trained model; the model used for (b), apart from incorporating attention cues, maintains the same structure and training configuration as the model used for (a). All three visualize the attention weights of the 8th attention head (head = 8) in the final layer of BERT, as this head has been observed to display the clearest representation among all heads

Hyperparameter SelectionPrototype Dimension Selection

In this section, we investigate the choice of prototype vector dimensionality. In our main contrastive experiments, we set the prototype vector dimension to 512. To comprehensively evaluate the impact of dimensionality on performance, we additionally include two commonly used settings from prior prototype-based method [14, 15] (64 and 256) and introduce a higher-dimensional configuration (1024). Unlike most previous works that employ Euclidean distance to measure semantic similarity, we adopt cosine similarity in this task. Compared to Euclidean distance, which is prone to the “curse of dimensionality” in high-dimensional spaces, cosine similarity focuses on directional alignment between vectors rather than their absolute positions or magnitude differences, thereby offering greater robustness in high-dimensional embedding spaces. Moreover, in low-dimensional spaces, cosine similarity may suffer from limited directional discriminability, leading to blurred class boundaries. As dimensionality increases, the angular degrees of freedom in the representation space also increase, allowing prototypes of different classes to be more widely dispersed on the unit hypersphere, thus enhancing inter-class separability.

Our experimental results corroborate this perspective. As shown in Table 7, across three named entity recognition datasets, higher dimensions generally yield better performance: when the dimensionality is reduced to 64, the model’s capacity to capture fine-grained semantic distinctions becomes relatively constrained; conversely, increasing the dimensionality from 512 to 1024 yields diminishing returns, with only marginal gains. Balancing model performance against computational efficiency, we consistently adopt a dimensionality of \(d = 512\) in both our main and ablation experiments. This choice fully leverages the directional sensitivity of cosine similarity while remaining practical for real-world deployment.

Table 7 Performance comparison under different prototype dimensions (F1 score in %).All experiments uniformly employ BERT-base as the encoder, with all other hyperparameters kept identical to those in the main experiments except for the dimensionality. Note: Bold values indicate the best performance \(\alpha \) and \(\beta \) Selection

The total objective function is as follows:

$$\begin \mathcal _} = \mathcal _} + \alpha \cdot \mathcal _} + \beta \cdot (\mathcal _} + \mathcal _}) \end$$

(24)

where \(\alpha \) and \(\beta \) are hyperparameters that balance the relative contributions of the prototype classification loss \(\mathcal _}\) and the combined contrastive and invariance loss \(\left( \mathcal _} + \mathcal _} \right) \) to the total objective. To better understand the sensitivity of our model to these weighting coefficients, we conduct a systematic ablation study over various \((\alpha , \beta )\) configurations.

As shown in Table 8, the performance trends are remarkably consistent across all three benchmark datasets (CoNLL2003, ACE2005, and GENIA). The optimal configuration is consistently achieved at \(\alpha = 1.0\) and \(\beta = 0.5\), yielding the highest F1 scores in every case. This suggests that our framework is robust to dataset domain shifts while benefiting from a fixed, well-tuned loss weighting strategy.

From the perspective of individual loss components, this finding aligns closely with our earlier ablation analysis: \(\mathcal _}\) plays a dominant role in shaping well-separated decision boundaries by explicitly supervising prototype assignments, whereas \((\mathcal _} + \mathcal _})\) encourages intra-class compactness and semantic invariance through contrastive learning and invariance regularization. Consequently, assigning a higher weight to \(\mathcal _}\)—as reflected in \(\alpha > \beta \)—effectively prioritizes discriminative power without compromising cluster cohesion, leading to superior overall performance.

Table 8 Performance comparison under different \(\alpha \) and \(\beta \) (F1 score in %).All experiments uniformly employ BERT-base as the encoder, with all other hyperparameters kept identical to those in the main experiments except for the dimensionality. Note: Bold values indicate the best performance.Low-resource Setting Result

To evaluate the effectiveness of our proposed prototype representation learning method under low-resource language settings, we simulate such conditions by constructing training sets with varying proportions of the full dataset. We also conduct experiments using Chen et al.’s Controlled Attention method as a primary baseline for comparison. The results are presented in Table 9.

Both our approach and the Controlled Attention method utilize boundary markers to construct attention cues; however, they differ significantly in their modeling mechanisms. While Chen et al. employ a bilinear module to encode contextual information and output a probability distribution over entity labels, our method concatenates special token vectors and directly learns class prototypes within the embedding space. Experimental results on the GENIA and CoNLL2003 datasets show that our method significantly outperforms Controlled Attention under low-resource conditions (with only 10% and 30% of the training data). Moreover, when 70% of the training data is retained, our model achieves performance close to that trained on the full dataset.

These advantages stem from the intrinsic properties of prototype-based learning: instead of relying on large amounts of labeled data to fit complex probability distributions, our method constructs a metric space where classification decisions are made based on semantic distances. In scenarios with limited annotations, this distance-based discriminative mechanism demonstrates greater robustness and generalization capability compared to models based on probabilistic estimation.

Table 9 All experiments were conducted using BERT-base under the same settings as in the main experiments. For training rates of 10% and 30%, we performed three independent runs and reported the average results to mitigate variance caused by random samplingPrototype Visualization AnalysisPrototype Stability Analysis

To analyze the stability of the prototype representation learning process, we employ Uniform Manifold Approximation and Projection (UMAP) [51,52,53], a dimensionality reduction and visualization technique that supports multiple distance metrics including cosine similarity, to visualize entity representations from the CoNLL2003 test set at training epochs 0, 2, 4, and 6. The results are shown in Fig. 4.

In the visualization, we mark the prototype center for each class and report two standard clustering quality metrics, the Silhouette Score [54] and the Davies-Bouldin Index (DBI) [55]. The Silhouette Score ranges from \([-1, 1]\), and values closer to 1 indicate more compact intra-class clusters and clearer separation between classes. The Davies-Bouldin Index reflects the ratio of within-cluster dispersion to between-cluster separation, with lower values indicating better clustering performance.

The results show that starting from epoch 2(seen Fig. 4b), the Silhouette Score gradually increases around 0.6, while the Davies-Bouldin Index exhibits a fluctuating downward trend. Visually, a coherent clustering structure emerges early in training, and subsequent epochs further refine this configuration. Throughout training, the overall cluster structure remains well separated without any sign of collapse. These findings suggest that the proposed prototype-based representation learning mechanism achieves strong stability and robust convergence, effectively inducing semantic clusters in the embedding space that are compact within classes and well separated across classes.

Fig. 4The alternative text for this image may have been generated using AI.

UMAP visualization of test set entity representations at training epochs 0, 2, 4, and 6. Labels 0–3 correspond to neg, org, misc, and per, respectively. Each subplot shows the clustering structure with colors indicating class labels and prototype centers marked as C0–C3. The Silhouette Score and Davies-Bouldin Index are shown in the title of each subplot. UMAP is configured with \(n\_neighbors=30\) (balancing local and global structure), \(min\_dist=0.1\) (yielding moderately compact clusters), and cosine distance as the metric

Intra-class Heterogeneity of NEG

In the main experiments, we adopt a unified prototype representation for negative examples. Although these negatives may encompass various types of boundary errors, we investigate the possibility of intra-class heterogeneity among them. As previously discussed, negative spans primarily arise from incorrect or composite entity boundaries. To explore this, we propose two hypotheses: (1) negatives can be grouped into two subtypes—those partially overlapping with true entity boundaries and those with no overlap; (2) negatives can be further divided into \(n+1\) subtypes, where n is the number of true entity classes—specifically, one subtype for each entity class corresponding to partial overlaps, plus an additional subtype for completely non-overlapping negatives.

In our experiments, we assign either 2 or \(n+1\) prototype vectors to negative examples, allowing them to self-cluster in the semantic space. We then visualize the resulting embeddings using UMAP and compute the Silhouette Score and Davies–Bouldin (DB) Index to compare against the baseline setting (\(m=1\), i.e., a single negative prototype). As shown in Fig. 5, both metrics perform worse under the multi-prototype configurations compared to \(m=1\). This suggests that, although semantic variations do exist among negative examples, the model is already capable of recognizing that none of them correspond to valid entities—rendering explicit modeling of negative subtypes unnecessary.

Fig. 5The alternative text for this image may have been generated using AI.

UMAP Visualization (Cosine) at Epoch 6 under Different NEG Clustering Configurations. Notably, when \(m = 1\), the clustering achieves the highest Silhouette Score and the lowest DB Index, outperforming both \(m = 2\) and \(m = n+1\), indicating more cohesive and well-separated clusters