LLM Hacking: Researchers Can Reconstruct User Prompts from Chatbot Responses

Researchers from IIT Bombay and Adobe Research have developed a method that can reconstruct the input prompts of large language models almost exactly from their text outputs, working without access to model weights and even transferring to other models. The method, called 'Previous-Token Prediction' (PTP), inverts the basic principle of language models by training an inverse model to predict previous tokens instead of the next token. This inverse model is trained from scratch solely using synthetic data generated by the target LLM. The inverse model can not only restore the original prompt exactly but also generate multiple semantically different prompt alternatives by varying decoding parameters. In a qualitative example from the paper, the prompt 'How to reach out to competitors to find their pricing strategies?' was exactly reconstructed, along with six other variants. Tests with real user prompts also showed semantically faithful reconstructions. A small inverse model trained on Qwen-3-0.6B-Chat could also reconstruct prompts from GPT-4o responses, capturing the underlying context and intent, meaning an attacker would not even need to know which model is behind a text output. This presents a broad security problem for companies, as proprietary system prompts with business secrets, moderation rules, or specialized instructions could be extracted, as well as confidential user inputs. The paper itself does not make explicit statements about attacks on commercial systems, but model makers need to clarify and potentially remediate this vulnerability.