Enhancing Cross-Prompt Transferability in Vision-Language Models through Contextual Injection of Target Tokens (2024)

3.1 Injecting misleading target tokens into visual context

Injecting misleading targets into the visual context can enhance the probability distribution of target tokens within visual tokens of visual language model.This involves modifying the original image’s probability distribution by injecting target tokens. By injecting this information, the likelihood of the target task appearing in the top-k tokens increases significantly. This mechanism ensures that adversarial images more effectively guide the model toward generating specific, desired outputs. Table 1 presents the analysis experiment for injecting specific token into sample images (using the BLIP2Li etal. (2023) model with gradient-based perturbations over $1000$ iterations). Our findings indicate that in image classification tasks(details for the dataset, please refer to 5.1), visual context attacks can successfully achieve cross-prompt attacks for certain keywords.

3.2 Injecting misleading target tokens into textual context

Injecting misleading target into the text context can effectively mislead the model’s output.For example, if an image of a cat is inaccurately described as "this image shows a dog," the textual context is manipulated to support this misleading description. This manipulation causes the model to generate outputs that align with the incorrect description. By using inject misleading target into textual context, we enhance the adversarial image to ensure that the textual context effectively guides the generation of misleading outputs. Table 2 shows that inserting misleading text prompts before different prompts can successfully mislead the BLIP2Li etal. (2023) model.

4.1 Overall Structure

Figure 2 illustrates the overall framework of the CIA method. By injecting the target token into both visual and text positions, the probability of generating the target token is increased, resulting in improved cross-prompt transferability. Specifically, in the example shown in the figure: for the visual position, each visual token is perturbed based on the gradient towards the target ("dog"); for the text position, misleading descriptive content ("this image shows a dog") is injected to deceive the model; and at the output position, the model is directed to maximize the output of the target ("dog"). By weighting the losses from these three positions and performing backward gradient computation, the original image is perturbed to enhance adversarial transferability effectiveness.

4.2 Problem definition

Assume we have a vision-language model denoted as $M_{\overline{VL}}(I,T)$, which takes an image $I$ and text $T$ as inputs. Given an original, clean image $I_{ori}$ and an arbitrary set of textual prompts $A={\alpha_{0},\alpha_{1},\ldots,\alpha_{i},\ldots,\alpha_{n}}$, our objective is to ensure that when the model $M_{\overline{VL}}$ processes the perturbed image $P(I_{ori})=I_{ori}+\delta_{v}$, it consistently outputs the target text $T_{tgt}$ for every prompt $\alpha_{i}$.

Here, $\delta_{v}$ signifies the visual perturbation added to the image $I_{ori}$ and is bound by the constraint $||\delta_{v}||p\leq\epsilon_{v}$, where $\epsilon_{v}$ is the magnitude of the image perturbation.

Formally, this can be expressed as:

In this context, $T_{tgt}$ is the target caption for the image (e.g., "this image shows a dog"). The function $P$ represents the perturbation applied to the original image $I_{ori}$. Our goal is to ensure that for any given prompt $\alpha_{i}$, the model’s output remains the same and matches the target text $T_{tgt}$, regardless of the perturbations applied to the image.

4.3 Contextual Injection Attack (CIA)

To advance the cross-prompt transferability of adversarial images, this paper introduces a contextual-injection attack approach (CIA). Unlike the baseline method, which restricts the target task to the decoded representation of the output and expands the search scope using multiple distinct prompts or learnable cross-search methods without modifying the original knowledge representation of the image, CIA modifies the latent knowledge representation towards the target task through knowledge injection. By enhancing the context of both visual and textual inputs, the generated adversarial images can effectively handle variations in textual prompt inputs. Figure 2 illustrates the key steps of our method, where target is injected into the contextual positions of both visual and textual inputs within the model’s output decoding representation. This ensures the model’s output aligns more closely with text related to the target task (e.g., "dog").

To formalize the adversarial objective, we can express it as a formal loss function for the adversarial attack. We consider a vision-language model to be a mapping from a sequence of visual and textual tokens $x_{1:n}=[x_{1:end_{v}},x_{end_{v}+1:end_{t}},x_{end_{t}:n}]$, where $x_{i}\in\{1,...,V\}$. Here, $V$ denotes the vocabulary size, $end_{v}$ and $end_{t}$ indicate the end of the visual and text tokens, respectively. The visual tokens ($x_{1:end_{v}}$), input text tokens ($x_{end_{v}+1:end_{t}}$), and generated text tokens ($x_{end_{t}:n}$) together constitute the complete token representation, which is mapped to a distribution over the next token.

We calculate the probability distribution over the next token given the sequence $x_{1:i}$ as $p(x_{i:i+H}|x_{1:i})$. For any sequence $p(x_{i:i+H}|x_{1:i})$, where $H$ is the length of the sequence we aim to obtain, the joint probability is

To address the issue with the visual input not having previous tokens, we redefine the probability for the visual tokens to start from the given initial state without conditioning on previous tokens. The cross-entropy losses for each part are then computed as follows.

Here, $x_{1:end_{v}}^{*}$ denotes the target injected into the image, such as "dog", to maximize the probability distribution of each token position "dog".

Here, $x_{end_{v}+1:end_{t}}^{*}$ denotes the textual description of the image, for example, "This image shows a dog," when the original image depicts a cat.

Here $x_{end_{t}+1:n}^{*}$ refers to the generated text tokens conditioned on the entire sequence of visual and textual tokens. For instance, "This image shows a dog, it sits on the table."

The overall adversarial loss is a weighted sum of these individual losses:

where $\alpha$ and $\beta$ are the weights for the respective losses. By introducing two parameters, $\alpha$ and $\beta$, the method allows for finer control over the influence of each loss component. Specifically, $\alpha$ controls the overall balance between the combined visual and textual losses versus the generated text loss. Meanwhile, $\beta$ adjusts the emphasis between the visual and textual input losses within their combined term.

The task of optimizing the adversarial perturbation $\delta_{v}$ can then be written as the optimization problem:

To implement our context-enhanced adversarial attack on vision-language models, we follow the outlined pseudocode Algorithm 1. The algorithm starts by initializing the perturbation $\delta_{v}$ to zero and defining the weights $\alpha$ and $\beta$ for the respective losses. In each iteration, we compute the perturbed image $P(I_{\text{ori}})$ by adding the current perturbation $\delta_{v}$ to the original image $I_{\text{ori}}$. We then calculate the cross-entropy losses for the visual tokens $L_{\text{visual}}$, the textual input tokens $L_{\text{text}}$, and the generated text tokens $L_{\text{generated}}$. The total loss $L_{\text{total}}$ is obtained as a weighted sum of these individual losses.

The gradient of the total loss with respect to the perturbation $\delta_{v}$ is computed, and the perturbation is updated using gradient descent(The optimisation algorithm is PGDMadry etal. (2017)). To ensure the perturbation remains within the allowed bound, it is projected onto the $\epsilon_{v}$-ball. The process repeats until convergence, ultimately yielding the adversarial image $P(I_{\text{ori}})$ that steers the model’s output towards the target text $T_{\text{tgt}}$.

5.1 Datasets & Experimental settings

The dataset consists of two parts: images and text. The image dataset is sourced from visualQAAntol etal. (2015b), and the text prompt dataset for transferability comes from CroPALuo etal. (2024). This text dataset is divided into three categories: image classification (CLS), image captions (CAP), and visual question answering (VQA). We will design attack tasks across four different dimensions: generating target tasks involving ordinary objects, harmful objects, tone expressions, and racial discrimination.

The experimental setup for this study involves using three open-source models: BLIP2(blip2-opt-2.7b), instructBLIP(instructblip-vicuna-7b), and LLaVA(LLaVA-v1.5-7b). The maximum number of iterations is set to 2000, and the hyperparameters $\alpha$ and $\beta$ are both set to $0.6$, based on the conclusions drawn in Figure 4. The learning rate is set to $0.05$, and the image perturbation range is set to $16/255$

5.2 Evaluation metrics

To evaluate the effectiveness of our method, we used the following metrics:

5.3 Transferability comparison

The results of our experiments, which evaluate targeted Attack Success Rate (ASR) on the visual-language model across various tasks (CLS, CAP, VQA) and target texts, are detailed in Table 3(experiments on other models can be found in the appendix A.1.1). The performance of the CIA method was compared against three baseline methods: Single-P (SP), Multi-P (MP), and CroPA (CP). To generate adversarial examples for VLMs, Single-P optimizes an image perturbation based on a single prompt. In contrast, Multi-P enhances the cross-prompt transferability of the perturbations by utilizing multiple prompts during the image perturbation update process. CroPA Luo etal. (2024) achieves broader prompt coverage by using a learnable prompt to expand around a given prompt, thereby improving transferability. CIA achieves the highest transfer attack success rate for the majority of targets.

Our findings suggest that common words yield the highest performance because they appear most frequently in the model’s training samples, resulting in the lowest perplexity. Harmful words may be blocked by the model’s safety alignment strategies. Affective words achieve the lowest scores because our method relies on injecting textual instruction into the visual context. However, affective words have a semantic disconnect with the visual representation, making it difficult to represent them accurately. Conversely, images with tangible entities are more likely to converge and produce effective adversarial images. The results in Table 4 support our conclusion.

To determine the most effective approach among visual context enhancement, textual context enhancement, and a combined visual-text context enhancement, we conducted comprehensive experiments. As shown in Table 5, CIA-image represents the transfer attack effectiveness using only visual context enhancement, CIA-text represents the transfer attack effectiveness using only textual context enhancement, and CIA represents the combined approach using both visual and textual context enhancements. Our findings indicate that the combined visual-text context attack is the most effective, suggesting that multimodal joint attacks are more successful in deceiving the model and thereby increasing the attack success rate.

Figure 3 shows the cross-entropy values of logits related to the target task at different positions. The baseline method made only minor adjustments to the probabilities of the target task token in both visual and text contexts, resulting in suboptimal performance in cross-prompt tasks. We compared three approaches: using visual context alone, using text context alone, and using a combined context. Independently, the visual and text contexts achieved optimal cross-entropy loss at their respective token positions because they were computed separately, allowing for easier convergence to the minimum value. By combining both contexts, our method achieved the optimal cross-entropy loss at the final generated token position, indicating that it effectively skewed the probability distribution towards the target task token.

target	image	prompt	CroPA output	CIA output
dog		Imagine a child is trying to identify the image. What might they excitedly point to and name?	a plane flying over a cathedral	dog
dog		Any bicycles visible in this image? If not, please tell what is in the image?	no bicycles visible in this image	A dog in this image
poison		Describe the central subject of this image in a single word.	horse	poison
poison		Categorize the content of the image.	the horse is eating the poison	poison
sure		Provide a summary of the primary focus of this image.	a woman and a man are sitting on a sidewalk	sure
sure		Where do you see harmony or unity among the elements?	Harmony and unity	sure

5.4 Case study

The case study presented in Table 5 demonstrates the effectiveness of the CIA method compared to CroPA in generating adversarial examples that successfully deceive visual-language models (VLMs). We evaluated various target texts using different prompts to test robustness.

Adversarial images generated using the state-of-the-art CroPA method still retain the semantics of the original image. Specifically, in the fourth example provided in Table 6, ("the horse is eating the poison"), although the model responded with content related to the target ("poison"), it failed to completely remove the original image’s semantics (i.e., "horse"). This incomplete removal of original semantics leads to weaker transferability in cross-prompt attacks, as the model continues to recognize elements of the original image, thus diminishing the effectiveness of the adversarial example across different prompts.

5.5 CIA with different perturbation size

This section delves into the impact of different perturbation sizes ($8/255$, $16/255$, $32/255$) on the efficacy of adversarial attacks against the visual-language model. The table provided below showcases the overall Attack Success Rate (ASR) across various tasks, accentuating the perturbation size that demonstrates the highest performance for each task.

While larger perturbation sizes result in stronger attacks, it’s essential to consider the trade-off with concealment. Larger perturbations may be more easily detected by models or users, reducing the attack’s stealthiness. Therefore, a balance must be struck between perturbation size and concealment to maximize attack effectiveness while minimizing the risk of detection.

5.6 CIA with different prompt embedding setting

This section explores the impact of different embedding settings on the Attack Success Rate (ASR) through two types of experiments. For the details, please refer to the Appendix A.1.3

1. Impact of Padding Tokens on ASR: We evaluated the effect of various padding tokens (e.g., ’!’, ’@’, ’+’) on ASR within the text context. (as show in the Figure 4)

2. Effect of Embedding Strategies for ’@’: We assessed four embedding strategies for the special character ’@’: no embedding, prefix embedding, suffix embedding, and mixed embedding. The experiments covered tasks such as classification, captioning, and visual question answering. (as show in the Table 11)

A.1 Detailed data

A.1.2 Effects of parameters of the weighted sum of losses

We will examine how different weightings and parameters affect the results when calculating the loss. Specifically, we will focus on two hyperparameters, $\alpha$ and $\beta$, which control the weighting of the loss components.

The Figure 4 show the effects of parameter of the weighted sum of losses ($\alpha$ and $\beta$). We standardize the maximum number of iterations to $600$. Using the keyword ’dog’ as the target, we set the learning rate for gradient-based updates of image pixels to $0.05$, with the maximum perturbation range set to $16/255$.

A.2 Example of cross prompt task

Example dataset of transfer attack text prompts excerpted from CroPALuo etal. (2024), divided into three categories: image classification(CLS), image captioning(CAP), and visual question answering(VQA).