Multi-Path Ensemble Detection of Prompt Injection Attacks via Embedding Similarity, Trajectory Analysis, and Fine-Tuned Classification

Abstract. Prompt injection attacks pose a critical threat to large language model (LLM) deployments, enabling adversaries to override system instructions, exfiltrate data, and bypass safety controls. We present a multi-path ensemble system that combines three complementary detection strategies: (1) centroid-based embedding similarity against curated attack pattern clusters, (2) trajectory analysis that detects multi-turn conversational escalation via velocity and acceleration metrics, and (3) a fine-tuned DeBERTa-v3 classifier trained on injection examples. Unlike prior work using weighted-average ensembles, we demonstrate that a simple max()-based aggregation preserves strong single-path signals and avoids the dilution problem inherent in averaging. Evaluated on three external datasets totaling 3,118 labeled samples, the system achieves 91.5% attack detection and 91.6% benign precision (composite score: 91.5). We identify weak categories — encoding attacks (60%), jailbreak variants (72%), and non-English inputs — and discuss the structural limitations that bound further threshold-based improvements.

1. Introduction

Large language models are increasingly deployed in production systems where they process untrusted user input alongside privileged system instructions. This creates an attack surface known as prompt injection, where adversarial inputs attempt to override the model’s intended behavior. Attacks range from simple instruction overrides (“ignore previous instructions”) to sophisticated multi-turn escalation sequences, roleplay-based jailbreaks, and encoded obfuscation techniques.

Existing defenses fall into several categories: input filtering via regular expressions or keyword matching, perplexity-based anomaly detection, dedicated classifier models, and embedding-based similarity approaches. Each has characteristic weaknesses. Keyword filters are brittle against paraphrasing. Classifiers trained on known attacks may miss novel attack vectors. Single-metric similarity approaches lack the multi-dimensional view needed to catch diverse attack strategies.

We propose a multi-path ensemble architecture that addresses these limitations by combining three orthogonal detection signals:

1. Centroid similarity — comparing input embeddings against pre-computed centroids representing known attack categories in a 384-dimensional embedding space.
2. Trajectory analysis — tracking the velocity, acceleration, and drift of conversation embeddings across turns to detect gradual escalation patterns.
3. Fine-tuned classification — leveraging ProtectAI’s DeBERTa-v3-base model trained specifically for prompt injection detection.

A key finding is that the ensemble aggregation strategy matters more than individual path accuracy. Replacing a weighted average with a max() operator — allowing any single path to independently trigger detection — yielded the largest single improvement during optimization, validating the principle that prompt injection detection benefits from OR-logic rather than consensus. Evaluated on 3,118 held-out samples from three external datasets, the system achieves 91.5% attack detection and 91.6% benign precision.

2. Related Work

Classifier-based detection. Inan et al. [1] and Rebedea et al. [2] train dedicated models to distinguish injections from benign input. ProtectAI’s DeBERTa-v3-base-prompt-injection-v2, which we use as one detection path, represents this approach. These classifiers achieve high accuracy on in-distribution attacks but may miss novel vectors.

Embedding similarity. Ayub and Majumdar [3] demonstrate that prompt injections cluster in embedding space, enabling detection via embedding-based classifiers trained on attack and benign prompts. Our centroid-based path extends this principle with multiple centroid sources (hand-crafted patterns, K-means clusters from real attacks, and task-specific additions) and uses cosine similarity to pre-computed centroids rather than supervised classifiers.

Multi-turn analysis. Russinovich et al. [4] introduce the Crescendo attack, demonstrating that multi-turn jailbreak attacks are particularly effective because they gradually escalate through benign-seeming dialogue turns. Our trajectory analysis path directly addresses this threat by measuring conversation dynamics — velocity, acceleration, and drift — rather than single-turn content.

Ensemble methods. Liu et al. [5] combine multiple classifiers via voting. We show that max()-based aggregation outperforms averaging and voting for injection detection, where a single strong signal from any path should be sufficient to trigger an alert.

Benchmarks. Several public benchmarks exist for evaluating injection defenses, including TensorTrust [6], BIPIA [7], AgentDojo [8], PurpleLlama CyberSecEval [9], and AdvBench [10]. Liu et al. [11] provide a formal framework for benchmarking prompt injection attacks and defenses.

3. System Architecture

The detection system processes each input through three independent paths, aggregates their signals via a max() ensemble, and produces a risk level classification. Figure 1 illustrates the architecture.

                        ┌─────────────────────────┐
                        │      Input Prompt        │
                        └────────────┬────────────┘
                                     │
                      ┌──────────────┼──────────────┐
                      │              │              │
                      ▼              ▼              ▼
              ┌──────────────┐ ┌──────────┐ ┌──────────────┐
              │   Path 1:    │ │  Path 2: │ │   Path 3:    │
              │  Centroid    │ │Trajectory│ │  DeBERTa-v3  │
              │  Similarity  │ │ Analysis │ │  Classifier  │
              │              │ │          │ │              │
              │ all-MiniLM   │ │ velocity │ │ protectai/   │
              │ -L6-v2       │ │ accel.   │ │ deberta-v3-  │
              │ 384-dim      │ │ drift    │ │ base-prompt- │
              │ embeddings   │ │ trend    │ │ injection-v2 │
              └──────┬───────┘ └────┬─────┘ └──────┬───────┘
                     │              │              │
                     ▼              ▼              ▼
              centroid_score  traj_score    classifier_score
                     │              │              │
                     └──────────────┼──────────────┘
                                    │
                                    ▼
                        ┌───────────────────────┐
                        │   Ensemble: max(...)   │
                        │                       │
                        │  ensemble_score =      │
                        │  max(centroid_score,   │
                        │      traj_score,       │
                        │      classifier_score) │
                        └───────────┬───────────┘
                                    │
                                    ▼
                        ┌───────────────────────┐
                        │   Risk Classification  │
                        │                       │
                        │  > 0.7  → critical    │
                        │  > 0.5  → high        │
                        │  > 0.35 → medium      │
                        │  else   → low         │
                        └───────────────────────┘

Figure 1. System architecture showing the three independent detection paths feeding into the max() ensemble aggregator and risk classifier.

3.1 Path 1: Centroid-Based Embedding Similarity

The centroid path encodes input text using the all-MiniLM-L6-v2 sentence transformer (384-dimensional output) and computes cosine similarity against a library of pre-computed centroids. Each centroid represents a prototypical attack pattern in embedding space.

Centroid sources. Centroids are assembled from three sources:

• Pattern-based centroids (8 categories): Hand-crafted example phrases for instruction override, roleplay jailbreak, hypothetical framing, authority manipulation, output manipulation, encoding obfuscation, context manipulation, and emotional exploitation. Each category centroid is the mean embedding of 4–8 representative phrases.
• Cluster-based centroids: K-means clustering (k=15) applied to embeddings of real jailbreak prompts from the HarmBench/MHJ dataset. Each cluster centroid captures a natural grouping of attack semantics.
• Task-specific centroids: Added iteratively during optimization — destructive_actions (12 patterns covering file deletion, database drops, etc.) and data_exfiltration (12 patterns covering credential theft, data leakage, etc.).

Score computation. Rather than using the maximum similarity to any single centroid (which is sensitive to outlier centroids), we use the mean of the top-3 centroid similarities as the aggregate centroid score:

$$\text{centroid\_score} = \frac{1}{3} \sum_{i=1}^{3} \text{sim}_{\text{top}_i}$$

This provides robustness against noisy centroids while still capturing strong matches.

Centroid curation. During optimization, we identified that cluster_8 — a K-means cluster containing 489 content/article-related samples — caused false positives on benign queries about article summarization. Removing this centroid reduced the false positive score from 0.516 to 0.428 while maintaining attack detection, demonstrating the importance of centroid quality over quantity.

3.2 Path 2: Trajectory Analysis

Single-turn analysis cannot detect multi-turn escalation attacks where each individual message appears benign but the conversation gradually drifts toward dangerous territory. The trajectory path addresses this by tracking the dynamics of embedding similarity over the conversation.

For each turn, we compute the maximum cosine similarity to any centroid and append it to a per-session history vector. We then derive the following metrics over a sliding window of size w = 5:

Velocity — the rate of change in similarity between consecutive turns:

$$v_t = s_t - s_{t-1}$$

Average velocity — mean velocity over the window:

$$\bar{v} = \frac{1}{|W|} \sum_{t \in W} v_t$$

Acceleration — rate of velocity change:

$$a_t = v_t - v_{t-1}$$

Trend score — exponentially weighted deviation from the window’s starting point, emphasizing recent movement:

$$\text{trend} = \sum_{t \in W} w_t \cdot (s_t - s_0), \quad w_t = \frac{e^{t/|W|}}{\sum e^{t/|W|}}$$

Volatility — standard deviation of velocities within the window, capturing erratic behavior.

Escalation alerts are triggered when metrics exceed empirically determined thresholds:

Table 1. Trajectory escalation alert thresholds.

The trajectory score for the ensemble is computed as:

$$\text{traj\_score} = \min(1.0,\; 2 \cdot \Delta s_{\text{total}}) + \min(0.3,\; v_t) \quad \text{if } \Delta s > 0$$

3.3 Path 3: Fine-Tuned Classifier

The third path uses ProtectAI’s deberta-v3-base-prompt-injection-v2, a DeBERTa-v3-base model fine-tuned for binary classification of prompt injections. The model outputs a probability of injection, which we use directly as classifier_score. This path provides an independent semantic signal trained on a different data distribution than our centroid library, offering complementary coverage.

The system degrades gracefully if the classifier is unavailable — the ensemble falls back to max(centroid_score, traj_score).

3.4 Ensemble Aggregation

The ensemble combines the three path scores via a max() operator:

$$\text{ensemble\_score} = \max(\text{centroid\_score},\; \text{traj\_score},\; \text{classifier\_score})$$

This design choice is motivated by the observation that prompt injection detection is fundamentally an OR-problem: an input should be flagged if any detection path identifies it as suspicious. A weighted average, by contrast, dilutes strong single-path signals — a centroid score of 0.8 averaged with a classifier score of 0.1 and trajectory score of 0.0 yields only 0.30, potentially falling below detection thresholds. The max() operator preserves the 0.8 signal.

This single architectural change produced the largest improvement in our optimization process (+7.6 percentage points), as detailed in Section 5.

3.5 Risk Classification

The ensemble score and alert metadata are mapped to a four-level risk classification:

Table 2. Risk level classification rules.

4. Evaluation Methodology

4.1 Development Set

We optimize on a curated development set of 70 prompts: 40 attack prompts spanning 12 categories and 30 benign prompts representing common user interactions. This set is intentionally small to enable rapid iteration; generalization is assessed separately via held-out evaluation (Section 4.3).

Attack categories (40 prompts):

Table 3. Attack categories and distribution in the development set.

Benign categories (30 prompts): Weather queries, programming help, education, recipes, travel planning, health, finance, creative writing, and general knowledge questions.

4.2 Scoring Metric

We use a composite score that weights attack detection more heavily than benign precision, reflecting the asymmetric cost of missed attacks versus false positives:

$$\text{composite} = \left(0.70 \times \frac{\text{attacks detected}}{\text{total attacks}} + 0.30 \times \frac{\text{benign clean}}{\text{total benign}}\right) \times 100$$

4.3 Held-Out Evaluation Datasets

To assess generalization beyond the development set, we evaluate on three external labeled datasets from HuggingFace, none of which were used during optimization:

Table 4. Held-out evaluation datasets. All datasets include both attack and benign labels, enabling measurement of both detection rate and false positive rate.

5. Experimental Results

5.1 Iterative Optimization

We used a small development set of 70 prompts (40 attacks, 30 benign) as a feedback signal during iterative optimization. This set is not representative of real-world performance — it served only to guide architectural decisions. We conducted 12 experiments; Table 5 summarizes the changes that were kept.

Table 5. Optimization timeline on the development set. Scores reflect dev set performance only; see Section 5.5 for held-out results. Delta column shows the relative impact of each change.

5.2 Negative Results and Discarded Experiments

Several experiments were informative despite failing to improve the score:

Table 6. Discarded experiments and failure analysis.

5.3 Contribution of Each Detection Path

The max() ensemble allows us to attribute detections to the path that provided the strongest signal:

• Centroid path: Primary detector for pattern-based attacks (instruction override, roleplay, hypothetical framing, destructive actions, exfiltration). Responsible for the majority of detections.
• Classifier path: Provides independent confirmation and catches semantic patterns not well-represented by centroids. Critical for encoding/obfuscation and indirect injection detection.
• Trajectory path: Activates on multi-turn escalation sequences where individual turns may score below single-turn thresholds. Complementary rather than primary.

5.4 The Structural False Positive Ceiling

During development, we observed that benign queries and weak attack prompts occupy the same centroid similarity band (~0.42–0.45). Experiment 12 confirmed this: reducing centroid influence enough to eliminate a known false positive also lost 5 true attacks. This represents an architectural ceiling that cannot be resolved by threshold tuning alone, since the all-MiniLM-L6-v2 embedding model was not trained to distinguish attacks from benign text. Held-out evaluation (Section 5.5) confirms this ceiling extends to a broader class of benign inputs.

5.5 Held-Out Evaluation Results

We evaluate the final system on 3,118 labeled samples from three external datasets not used during optimization.

Table 7. Held-out evaluation results on external datasets.

The overall composite score is 91.5, with 91.5% attack detection and 91.6% benign precision across 3,118 samples.

Per-category analysis on the neuralchemy dataset reveals where the system is weakest:

Table 8. Per-category detection rates on the neuralchemy held-out set, showing weakest categories.

False positive analysis. The 156 false positives across all held-out datasets cluster around three patterns: (1) benign prompts containing security-adjacent language (“cybersecurity awareness video”, “ethical guidelines”), (2) instructional text with imperative phrasing (“replace all sentences”, “generate a question about”), and (3) prompts referencing AI behavior (“as an AI language model, you must…”). These patterns overlap with attack centroids in embedding space, confirming the structural ceiling identified in Section 5.4 extends to a broad class of real-world benign inputs.

The deepset gap. The deepset dataset yielded only 45% attack detection — significantly below the other datasets. Analysis of the 33 missed attacks reveals three causes: (1) non-English injections (German text scored 0.114, below any threshold), (2) subtle social-engineering prompts that lack explicit injection markers (“act as an interviewer”), and (3) long-form attacks where the injection payload is diluted by surrounding benign context. These represent genuine blind spots rather than threshold miscalibration.

6. Design Insights

6.1 OR-Logic Outperforms Consensus for Security Detection

The most impactful architectural finding is that max() aggregation outperforms weighted averaging. Security detection is fundamentally asymmetric: a false negative (missed attack) is far costlier than a false positive. Weighted averaging assumes all paths should agree, which dilutes confident single-path detections. The max() operator implements OR-logic — any path can independently trigger an alert — which matches the threat model.

6.2 Centroid Quality Outweighs Quantity

Adding a single targeted centroid (destructive_actions, 12 patterns) was the largest individual improvement during optimization, while removing a noisy centroid (cluster_8, 489 samples) reduced false positive scores. The lesson: curated, specific centroids are more valuable than large, undifferentiated clusters.

6.3 Trajectory Matters for Multi-Turn Attacks

While trajectory analysis is not the primary detector, it provides a unique capability: detecting attacks that unfold gradually across turns. A three-message escalation sequence (debug access → bypass permissions → delete files) may have each message score below single-turn thresholds, but the trajectory path detects the sustained velocity toward danger.

6.4 Systematic Experimentation Prevents Wasted Effort

The 12-experiment optimization process, tracked with per-experiment scoring, prevented several intuitively appealing but ultimately harmful changes. Experiment 3 (using max_similarity instead of top-3 average) appeared promising but produced no improvement — the real bottleneck was the ensemble aggregation, not the centroid scoring. Without systematic tracking, this root cause might have been missed.

7. Implementation

The system is implemented in Python using Flask for the REST API. Key dependencies include:

• Sentence Transformers (all-MiniLM-L6-v2) for 384-dimensional text embeddings
• scikit-learn for cosine similarity computation and K-means clustering
• Transformers (HuggingFace) for the ProtectAI DeBERTa-v3 classifier
• PyTorch for model inference
• NumPy for trajectory metric computation

The system exposes a REST API with endpoints for analysis (/api/analyze), session management, centroid inspection, and configuration. A web frontend provides real-time visualization of similarity scores, trajectory charts, and alert history.

Deployment is containerized via Docker, with centroids persisted as NumPy .npz files for fast loading.

7.1 Detection Configuration

Table 9. Production detection thresholds.

8. Limitations and Future Work

Encoding and jailbreak coverage. Held-out evaluation reveals 60% detection on encoding attacks and 72% on jailbreak variants — the weakest categories. These require dedicated centroid coverage or classifier fine-tuning to address.

Embedding model ceiling. The all-MiniLM-L6-v2 model was not trained to distinguish attacks from benign text, leading to structural overlaps (Section 5.5). Fine-tuning the embedding model for this domain is the most promising avenue for improvement.

Language coverage. The current system is English-only, as both the embedding model and centroid patterns are English-language. Cross-lingual attacks represent an unaddressed threat vector.

Adaptive adversaries. We do not evaluate against adversaries who have knowledge of our detection system. Adversarial robustness testing — crafting inputs that specifically evade centroid similarity, trajectory tracking, and the classifier — is critical future work.

Trajectory cold start. The trajectory path requires multiple turns to build a meaningful history. Single-turn attacks bypass this path entirely, though they are typically caught by the centroid or classifier paths.

Future directions include: (1) fine-tuning the sentence transformer on attack/benign separation, (2) adversarial robustness evaluation, (3) cross-language support, (4) integration with downstream task execution monitoring, and (5) online centroid updates from newly observed attacks.

9. Conclusion

We presented a multi-path ensemble system for prompt injection detection that combines embedding similarity, trajectory analysis, and fine-tuned classification. The key architectural insight — using max() aggregation instead of weighted averaging — confirms that injection detection benefits from OR-logic that preserves strong individual signals. Evaluated on 3,118 held-out samples from three external datasets, the system achieves 91.5% attack detection and 91.6% benign precision. Weak spots remain in encoding attacks (60%), jailbreak variants (72%), and non-English inputs — all traceable to the general-purpose embedding model’s inability to separate attack and benign semantics in these categories. The trajectory analysis component adds a unique dimension by detecting multi-turn escalation patterns invisible to single-turn analysis. While limitations remain in embedding model resolution, cross-lingual coverage, and adversarial robustness, the multi-path approach provides a practical foundation for production prompt injection defense.

References

1. H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, and M. Khabsa. Llama Guard: LLM-based input-output safeguard for human-AI conversations. arXiv preprint arXiv:2312.06674, 2023.
2. T. Rebedea, R. Dinu, M. Sreedhar, C. Parisien, and J. Cohen. NeMo Guardrails: A toolkit for controllable and safe LLM applications with programmable rails. arXiv preprint arXiv:2310.10501, 2023.
3. M. A. Ayub and S. Majumdar. Embedding-based classifiers can detect prompt injection attacks. arXiv preprint arXiv:2410.22284, 2024.
4. M. Russinovich, A. Salem, and R. Eldan. Great, now write an article about that: The Crescendo multi-turn LLM jailbreak attack. arXiv preprint arXiv:2404.01833, 2024.
5. Y. Liu, G. Deng, Z. Li, K. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, and Y. Liu. Prompt injection attack against LLM-integrated applications. arXiv preprint arXiv:2306.05499, 2023.
6. S. Toyer, O. Watkins, E. A. Menber, J. Svegliato, L. Bailey, T. Wang, I. Dunn, S. Russell, and S. Emmons. Tensor Trust: Interpretable prompt injection attacks from an online game. ICLR, 2024. arXiv:2311.01011.
7. J. Yi, Y. Xie, B. Zhu, E. Kiciman, G. Sun, X. Xie, and F. Wu. Benchmarking and defending against indirect prompt injection attacks on large language models. arXiv preprint arXiv:2312.14197, 2023.
8. E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramer. AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. arXiv preprint arXiv:2406.13352, 2024.
9. M. Bhatt, S. Chennabasappa, C. Nikolaidis, S. Wan, I. Evtimov, D. Gabi, D. Song, F. Ahmad, C. Aschermann, et al. Purple Llama CyberSecEval: A secure coding benchmark for language models. arXiv preprint arXiv:2312.04724, 2023.
10. A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023.
11. Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong. Formalizing and benchmarking prompt injection attacks and defenses. In Proceedings of the 33rd USENIX Security Symposium, 2024. arXiv:2310.12815.