Ph.D. Candidate · Florida State University

Building safer
multimodal AI
systems.

I investigate how vision-language models fail — geometrically, semantically, and adversarially — to make them more robust, interpretable, and trustworthy.

Representation Learning Adversarial Robustness Mechanistic Interpretability Vision-Language Models
Get in touch GitHub Google Scholar
Publications
7 papers
Montasir Shams
Research since
2021
Research focus
What I work on

My research sits at the intersection of representation geometry, adversarial machine learning, and multimodal AI safety. I study how vision-language models encode information — and crucially, how that encoding breaks down.

Current work targets adversarial vulnerabilities in BLIP-2, LLaVA-1.5, and Qwen-VL, probing teacher-forcing gaps, gradient dilution, and Q-Former bottleneck effects through a mechanistic interpretability lens.

Earlier work revealed surprising geometric properties of ViT embedding spaces: equivalence structures, inverted LID behavior, and cross-modal semantic misalignment — findings with direct implications for model safety and robustness evaluation.

Adversarial attacks on VLLMs
Teacher-forcing exploitation, gradient dilution, attention knockout in BLIP-2, LLaVA-1.5 & Qwen-VL
ViT embedding geometry
Equivalence structures, local Lipschitz estimation, intrinsic dimensionality (LID), Jacobian/SVD curvature
Mechanistic interpretability
Attention knockout, information flow analysis, cross-modal alignment failure in multimodal transformers
Representation vulnerability
Semantic misalignment, noise-voting detection, adversarial robustness evaluation in shared embedding spaces
Latest
News & updates
Apr 2026
Paper accepted at IEEE WCCI 2026 Accepted — "Intriguing Equivalence Structures of the Embedding Space of Vision Transformers"
2025
Two papers accepted at IJCNN 2025 Accepted on zero-shot evaluation gaps and cross-modal misalignment in VLMs
2024
Paper accepted at IROS 2024 Oral — vision-language navigation attacks via representation exploitation
ongoing
Active work on adversarial attacks for BLIP-2, LLaVA-1.5, and Qwen-VL via teacher-forcing exploitation and attention knockout WIP
Selected work
Publications

* equal contribution  ·  Montasir Shams in bold

2026
Intriguing Equivalence Structures of the Embedding Space of Vision Transformers
Shaeke Salman*, Montasir Shams*, Chashi Mahiul Islam, Mao Nishino, Xiuwen Liu
WCCI 2026IEEE World Congress on Computational Intelligence
2025
Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models
Shaeke Salman*, Montasir Shams*, Xiuwen Liu, Lingjiong Zhu
IJCNN 2025International Joint Conference on Neural Networks
2025
Unaligning Everything: Or Aligning Any Text to Any Image in Multimodal Models
Shaeke Salman, Montasir Shams, Mao Nishino, Xiuwen Liu
IJCNN 2025
2024
Malicious Path Manipulations via Exploitation of Representation Vulnerabilities of Vision-Language Navigation Systems
Chashi Mahiul Islam*, Shaeke Salman*, Montasir Shams, Xiuwen Liu
IROS 2024IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems
2022
Learning Tabular Embeddings at Web Scale  ·  Towards Unveiling Dark Web Structured Data
Montasir Shams, Sophi Pavia, Rituparna Khan, Anna Pyayt, Michael Gubanov
IEEE BigData 20222 papers
review
On the Representation Vulnerability of Vision Transformers
Montasir Shams, Shaeke Salman, Chashi Mahiul Islam, Mao Nishino, Xiuwen Liu
IEEE TAI — Under Review
Jacobian/SVD curvature analysis, local Lipschitz estimation, noise-voting detection; inverted LID behavior as geometric characterization of representation matching attacks.
WIP
Adversarial Attacks on Vision-Language Large Models via Teacher-Forcing Exploitation
Montasir Shams et al.
Work in Progress
Gradient dilution and language-prior dominance in BLIP-2, LLaVA-1.5-7B, and Qwen-VL; mechanistic interpretability via attention knockout.
Background
Experience & education
Research & teaching
Graduate Research Assistant2021 – present
Deep Learning & CV/NLP Lab, FSU
Adversarial attacks on VLMs, ViT embedding geometry, representation vulnerability, mechanistic interpretability.
Graduate Teaching Associate2022 – present
Department of CS, FSU
Data Science, Python Programming, Database Systems
Lecturer2017 – 2021
Daffodil International University, Bangladesh
AI, Database Systems, Python, Compiler Design
Education
Ph.D. in Computer Science2021 – present
Florida State University
Advisor: Dr. Xiuwen Liu
M.S. in Computer Science2021 – 2024
Florida State University
GPA: 3.75 / 4.00
B.Sc. in Computer Science & Engineering2013 – 2017
KUET, Bangladesh
GPA: 3.76 / 4.00
Toolkit
Skills
Languages & Frameworks
Python PyTorch HuggingFace NumPy OpenCV CUDA
Models & Architectures
CLIP ViT BLIP-2 LLaVA Qwen-VL Mask2Former
Research Methods
Adversarial attacks Jacobian/SVD LID estimation Attention knockout Embedding geometry