Utshab Kumar Ghosh
PhD student, Missouri S&T: information retrieval, agentic search, LLM evaluation
I study the gap between what looks relevant and what actually helps in search and RAG. I have first-author papers accepted at ACM SIGIR, ICTIR and CIKM, and nearly three years of industry ML work, including production RAG systems for government.
News
- Our SAFE-SEARCH workshop was accepted at ECIR 2027.
- First-author paper accepted at ACM CIKM 2026.
- First-author paper accepted at ACM ICTIR 2026.
- First-author reproducibility paper accepted at ACM SIGIR 2026.
- Our Tip-of-the-Tongue paper appeared in the TREC 2025 proceedings.
Publications
Conference papers, preprints and journal articles. † means equal contribution.
Conference papers
Entity Labels Are Not Entity Signals: A Framework for Observable Relevance in Document Re-Ranking
Preprints
Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG
Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search
Journal articles
Peer reviewer: FIRE, LREC, GRAIL. Co-organizer: SAFE-SEARCH workshop (Safety, Accountability, Faithfulness, and Evaluation in Conversational Search), ECIR 2027.
Experience
Where I have worked and studied.
Work
Dec 2022 - Jun 2025 · Dhaka, Bangladesh
Machine Learning Engineer
Nascenia Ltd.- Architected a production RAG pipeline with LangChain, LlamaIndex and ChromaDB. Did R&D on other LLMs such as Mistral and Bloom before deploying Llama-3 for document Q&A.
- Deployed it on AWS with GPU acceleration, handling 10,000+ daily requests and improving citizen service response times.
- Fine-tuned and quantized Llama-3.1 8B models (5-bit), cutting cloud inference costs by almost 70%. Checked answer quality against the full model on real user questions first.
- Integrated the AI core into a secure Laravel web application, with MySQL-backed dashboards and admin panels for government use.
- Led projects with teams of 4-10 developers. Managed client relationships with the Government of Bangladesh and international clients in Turkey and Sweden.
May 2019 - Jun 2025 · Dhaka, Bangladesh
Co-Founder & Mentor
Code Studio / Code Studio Academy- Co-founded a tech startup and grew it from 2 to 15 people across 100+ client projects.
- Designed an ML bootcamp that trained 120+ students and placed 30+ graduates at companies in Bangladesh.
Oct 2020 - Sep 2022 · Florida, USA (Remote)
AI Research Assistant
MyMedicalHUB Int.- Developed a computer vision system for Parkinson’s disease detection by gait analysis.
Education
Aug 2025 - Expected Aug 2028 · Rolla, MO, USA
Ph.D. in Computer Science
Missouri University of Science & Technology- Graduate Research Assistant
- Teaching Assistant (Algorithms)
- Supervisor: Dr. Shubham Chatterjee
- Research: Information Retrieval, Evaluation, Agentic Search, RAG
- CGPA 4.0/4.0
Jan 2017 - Oct 2022 · Rajshahi, Bangladesh
B.Sc. in Computer Science & Engineering
Rajshahi University of Engineering & Technology- Thesis: Most Dominant Metabolomic Biomarkers Identification for Lung Cancer
Projects
Research first. Code links where the code is public.
TREC 2026 AutoJudge: LLM Judges for Ranking RAG Systems
Ranking RAG systems needs human assessors, and that does not scale. An automatic judge has to rank systems the way people would, with no human answer keys. We built an LLM nugget-recall judge (Qwen3-30B and gpt-oss-20b served with vLLM) and an LLM-free consensus judge with leave-one-team-out pooling.
- Added validity checks that caught six silent pipeline failures.
- Fixed reasoning-model truncation. Unreadable judgments fell from 12.4% to 2.7%.
Diagnosing Evidence Loss in Agentic Search
When a search agent fails, retrieval and final-answer metrics cannot say why. It may have asked the wrong query, the retriever may have missed the evidence, or the agent may have ignored what it found. This stage-wise diagnostic uses about 173,000 passage-omission counterfactual replays over HotpotQA and MuSiQue, with a ReAct agent on BM25 + dense + RRF + cross-encoder retrieval.
- Later-hop query formulation is a major bottleneck. Gold-passage coverage rose from 43.8% to 69.8% with gold queries.
- 89% of residual misses happen before reranking.
- 68-76% of process failures are invisible to final-answer F1.
In progress
Robust-EO: An Entity-Oriented Retrieval Benchmark on Robust04
Existing retrieval benchmarks lack entity-centric annotations, so entity-aware retrieval is hard to evaluate cleanly. Building an entity-oriented benchmark on Robust04 with centrality-aware document annotations, direct topic-entity judgments, and aligned entity- and document-ranking tasks.
Biomedical RAG for Stem Cells
LLMs alone can produce unsupported claims in specialized fields like stem cell research. This retrieval-augmented system integrates biomedical corpora with LLM generation to produce citation-backed research answers.
Other
Asguard: Car & Home Maintenance
I built and shipped this app for iPhone and Android. It tracks car and home maintenance and reminds you before anything is overdue.
- Loads factory service schedules for 94 popular models from 17 makers, and every one is editable.
- Logs costs, receipts and fuel economy, with charts of what each car and home costs.
- Works fully offline, with no account, no servers and no network requests.
- Built with Flutter and Dart, using Riverpod for state and Drift for on-device storage, with 91 test files.
More on GitHub
- ML Competitions & Practice: solutions from Kaggle, HackerEarth and other ML competitions.
- Competitive Problem Solving: 300+ solved problems from programming contests.
- ML & DS Bootcamp: the machine learning and data science curriculum behind the bootcamp.
Skills
The tools and methods I work with, grouped by what I use them for.
Programming
- Python
- SQL
- C/C++
Stats & Experimentation
- Hypothesis testing
- Paired bootstrap
- Multiple-comparison correction
- Rank correlation and agreement
- Counterfactual analysis
ML and IR
- Machine Learning
- Deep Learning
- Natural Language Processing (NLP)
- Large Language Models (LLMs)
- Retrieval-Augmented Generation (RAG)
- LLM Agents
- Information Retrieval
- Re-ranking
- Query Reformulation
- Retrieval and LLM Evaluation
- Learning to Rank
- Dense, Late-Interaction and Hybrid Retrieval
- LLM-as-a-Judge
- Fine-tuning
- Prompt Engineering
- Quantization
- Multi-GPU Training
Frameworks and Tools
- PyTorch
- TensorFlow
- Hugging Face
- HF Accelerate
- PyTerrier
- Pyserini
- FAISS
- LangChain
- LlamaIndex
- Pandas
- NumPy
- Ollama
- vLLM
Infrastructure
- AWS
- Docker
- Git
- Linux
- FastAPI
- MySQL
- ChromaDB
- Redis
- MongoDB
Competitions & Awards
Results from ML and IR competitions.
Running
Paper in proceedings
read the paper
Top 2.6%
54th of 2034 · leaderboard
Top 3%
77th of 2796 · leaderboard
Awards
The Swedish Institute Scholarship for Global Professionals
2025 · Awarded to 300 scholars worldwideKaggle Expert (x3)
Kaggle profile300+ problems solved
Codeforces, CodeChef, LeetCodeGold Medalist
National High School Programming ContestTestimonials
What colleagues and collaborators say.
Utshab is a polite yet cheerful person with a passion for music and computer science. He is hard-working, keeping his programming fundamentals strong through competitive programming while also working on machine learning, Django, PyGame and more. As co-founder of the software startup Code Studio, he has proven his leadership and development skills. His skill set can be an asset to any company that makes the best use of him.
Tahsin Masrur
Software Engineer, Google
During Utshab’s tenure as a smartbook writer at 10 Minute School, he was frequently assigned responsibilities for writing and reviewing smartbooks. He was a quick learner, and there were numerous occasions where he proved his skill at both independent and collaborative work. His proactive and self-motivated nature made him a valuable asset to our team. He is a well-spoken and intellectual individual.
I have known Utshab for a long time. In those years, I have seen him excel not only at the core elements of data science and machine learning, but also at skills and tasks that extend well beyond the scope of his role. His sense of responsibility and his mastery of both can be critical to any team’s success. I have only optimistic predictions for his career trajectory.
Mahir Ashef
Co-Founder, Qubitech Solutions
Utshab is a hard-working person, enthusiastic about learning new things, with profound knowledge of machine learning and data science, which he shares with aspiring data scientists through his ML Bootcamp program. He worked with me at Code Studio, where his leadership stands out. He is a good mentor, a creative person and, above all, a helpful friend.
Soykot Hosen
Founder & CEO, Code Studio
Get in Touch
Email is the best way to reach me. I am open to research collaborations. Outside of work I am a musician.
Location
Rolla, MO, USA