International Journal on Science and Technology

E-ISSN: 2229-7677     Impact Factor: 9.88

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Call for Paper Volume 17 Issue 2 April-June 2026 Submit your research before last 3 days of June to publish your research paper in the issue of April-June.

Privacy-Preserving Legal Assistance: An Offline RAG-Based Large Language Model for the Indian Legal Context

Author(s) Mr. Abhishek Kumar Singh, Dr. Md Sajid Anwer
Country India
Abstract In this AI progressing World, there is a lot of legal work which can be done with the help of AI (LLM) with rapid rate and minimal human effort which makes it perfect for legal sector, but there is issue with how maximum LLM model works or integrated. Currently most of the LLMs are dependent on cloud-based system to operate which introduces data privacy risks for user also most of the LLMs are trained on the Western legal Dataset which introduces jurisdictional bias with respect to Indian laws. And third but very important issue with generalized models is “hallucination”. LLM models are trained in a way that they have to respond to the query confidently which makes it unreliable for domain specific work like Indian Constitution and Bharatiya Nyaya Sanhita (BNS). To handle these critical issues of LLMs in legal sector, this paper proposes a fully localized, offline architecture that work with the integration of RAFT (RAG + Fine Tuning) with 8-billion parameter Llama-3 model and Chroma DB pipeline. By implementing hallucination shield and Fine Tuning our model achieved a 95% retrieval accuracy on Indian penal statues and successfully blocked 75% of out of the bounds queries as compare to base model, which makes it perfect framework for privacy preserving offline legal AI.
Keywords Legal AI, LLM, RAG, RAFT, Llama3, Chroma DB, Indian Legal Context, GROQ, BNS, Indian Constitution
Field Engineering
Published In Volume 17, Issue 2, April-June 2026
Published On 2026-04-04

Share this