Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Posts
Future Blog Post
Published:
This post will show up by default. To disable scheduling of future posts, edit config.yml and set future: false.
Blog Post number 4
Published:
This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.
Blog Post number 3
Published:
This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.
Blog Post number 2
Published:
This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.
Blog Post number 1
Published:
This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.
portfolio
Portfolio item number 1
Short description of portfolio item number 1
Portfolio item number 2
Short description of portfolio item number 2
publications
Self-Calibrating Scene Understanding Based on Motifnet
Published in Chinese Conference on Pattern Recognition and Computer Vision (PRCV) 2019, 2019
We propose a self-calibrating approach to scene understanding that leverages Motifnet to jointly reason about object relationships and spatial context. The method enables adaptive calibration of scene graph generation without requiring explicit calibration supervision, improving relational inference in complex visual scenes. Experiments demonstrate that the self-calibration mechanism leads to more coherent and accurate scene representations across diverse visual environments.
Temple: Learning Template of Transitions for Sample Efficient Multi-task RL
Published in AAAI Conference on Artificial Intelligence 2021, 2021
Transferring knowledge among various environments is important for efficiently learning multiple tasks online. Most existing methods directly use previously learned models or optimal policies to learn new tasks, but these may be inefficient when the underlying models or optimal policies are substantially different across tasks. We propose Template Learning (TempLe), a PAC-MDP method for multi-task reinforcement learning applicable to tasks with varying state/action spaces without prior knowledge of inter-task mappings. TempLe gains sample efficiency by extracting similarities of transition dynamics across tasks even when their underlying models or optimal policies have limited commonalities. We present two algorithms for an online and a finite-model setting respectively, proving that TempLe achieves much lower sample complexity than single-task learners or state-of-the-art multi-task methods. Systematic experiments show that TempLe universally outperforms state-of-the-art multi-task methods in various settings and regimes.
ODE4ViTRobustness: A Tool for Understanding Adversarial Robustness of Vision Transformers
Published in Software Impacts 2023, 2023
We present ODE4ViTRobustness, a software tool for analysing and understanding the adversarial robustness of Vision Transformers (ViTs). The tool leverages ordinary differential equation perspectives on ViT depth dynamics to provide structured robustness evaluation, enabling practitioners to probe how attention-based architectures respond to adversarial perturbations. ODE4ViTRobustness provides a unified interface for benchmarking multiple ViT variants and attack strategies, contributing to the broader effort of building interpretable and trustworthy vision models.
DIMBA: Discretely Masked Black-Box Attack in Single Object Tracking
Published in Machine Learning (Springer) 2024, 2024
We develop a method to generate adversarial perturbations for single object tracking under black-box settings, where perturbations are added only on initialized frames rather than throughout entire videos. Our approach uses reinforcement learning to identify critical frame patches while minimizing computational overhead. We evaluate DIMBA on both long-term and short-term datasets—OTB100, VOT2018, UAV123, and LaSOT—and demonstrate effectiveness across three tracker types: discrimination-based, Siamese-based, and reinforcement learning-based models. We release DIMBA as an open-source tool to facilitate further research into neural network vulnerability and robustness in object tracking.
Representation-Based Robustness in Goal-Conditioned Reinforcement Learning
Published in AAAI Conference on Artificial Intelligence 2024, 2024
We study adversarial robustness in goal-conditioned reinforcement learning (GCRL), an area previously unexplored. We first demonstrate that attacks and robust representation training methods designed for traditional RL become less effective when applied to GCRL. We then introduce a Semi-Contrastive Representation attack that requires only policy function information and operates during deployment, along with Adversarial Representation Tactics that combine adversarial augmentation with regularization to strengthen agent robustness against perturbations. We validate our methods across multiple state-of-the-art GCRL algorithms and release the ReRoGCRL toolkit publicly.
Tiny Refinements Elicit Resilience: Toward Efficient Prefix-Model Against LLM Red-Teaming
Published in arXiv preprint arXiv:2405.12604, 2024
We present a plug-and-play prefix module that reconstructs input prompts using fewer than 30 additional tokens to mitigate toxic outputs from large language models. The sentinel model addresses parameter inefficiency and limited model accessibility for fine-tuning large target models. We employ interleaved training using Proximal Policy Optimization to jointly optimize both red team and sentinel models, incorporating a value head-sharing mechanism inspired by multi-agent centralized critic approaches. Testing across text-to-text and text-to-image applications demonstrates effectiveness against larger models including Llama-2, GPT-3.5, and Stable Diffusion, positioning the framework as a practical safety enhancement for a wide range of applications.
Boosting Adversarial Training via Fisher-Rao Norm-based Regularization
Published in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024, 2024
We address the challenge of maintaining standard generalization performance in adversarially-trained neural networks. Using the Fisher-Rao norm as a geometrically invariant complexity metric, we derive bounds on Rademacher complexity for ReLU-activated networks and identify a complexity variable that correlates with the generalization gap between adversarial and standard training. Based on these theoretical insights, we introduce Logit-Oriented Adversarial Training (LOAT), a regularization framework that reduces the robustness-accuracy trade-off with minimal computational overhead. LOAT demonstrates consistent improvements across multiple established adversarial training algorithms including PGD-AT, TRADES, MART, and DM-AT on various architectures.
Continuous Geometry-Aware Graph Diffusion via Hyperbolic Neural PDE
Published in European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD) 2024, 2024
We address scalability limitations in Hyperbolic Graph Neural Networks (HGNNs) by reformulating information propagation as a continuous partial differential equation. Treating network depth as temporal evolution and using node-wise attention mechanisms as diffusivity on non-Euclidean manifolds, we introduce the Hyperbolic Graph Diffusion Equation (HGDE)—a flexible vector flow function that can be integrated to obtain expressive hyperbolic node embeddings. HGDE handles both low- and high-order proximity through local-global diffusivity functions. Experiments on node classification, link prediction, and image-text classification demonstrate that the proposed method consistently outperforms competitive models by a significant margin.
Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving
Published in arXiv preprint arXiv:2410.12568, 2024
We present RAPID, a framework that integrates large language models into reinforcement learning for autonomous driving. RAPID combines three key elements: using offline data from an LLM-based driving agent to distill expert knowledge into faster RL policies, introducing robust distillation techniques to preserve both performance and robustness from the LLM teacher, and employing a mix-of-policy strategy with a policy adapter for joint decision-making during online fine-tuning. Extensive experiments demonstrate that RAPID successfully integrates LLM knowledge into scaled RL policies in a way that is efficient, adaptable, and robust while reducing knowledge loss under distribution shift.
Interpreting Safety: A LLM and STPA Approach
Published in Pacific Rim International Conference on Artificial Intelligence (PRICAI) 2025, 2025
We combine large language model reasoning with Systems-Theoretic Process Analysis (STPA) to improve interpretability and structured safety assessment in AI systems. By leveraging LLMs to automate and augment the identification of unsafe control actions and loss scenarios defined by STPA, our approach produces interpretable safety analyses that bridge formal safety engineering and modern machine learning. Evaluations demonstrate that the LLM-guided STPA workflow surfaces hazards more comprehensively than manual analysis while remaining accessible to practitioners without deep safety engineering expertise.
A Black-Box Evaluation Framework for Semantic Robustness in Bird’s Eye View Detection
Published in AAAI Conference on Artificial Intelligence 2025, 2025
We address robustness concerns in camera-based Bird’s Eye View (BEV) perception systems used in autonomous driving. Our black-box evaluation framework adversarially optimizes three semantic perturbations—geometric transformation, color shifting, and motion blur—to test model vulnerability. We introduce a smoothed distance-based surrogate function to replace the mAP metric and present SimpleDIRECT, a deterministic optimisation algorithm that utilises observed slopes to guide the optimisation process. Benchmarking ten recent BEV models, we find that PolarFormer demonstrates superior robustness while BEVDet shows significant vulnerability, with precision reduced to zero under adversarial perturbations.
Probabilistic Certification of Non-Toxicity in Vision-Language Models via Visual Embedding Smoothing
Published in arXiv preprint arXiv:2503.10661, 2025
Vision-Language Models (VLMs) are increasingly deployed in safety-critical applications but remain susceptible to jailbreak attacks that induce toxic outputs. While existing defence mechanisms for VLMs, such as model fine-tuning and response evaluation, have shown empirical effectiveness, they are predominantly heuristic, lack theoretical non-toxicity guarantees, and can be bypassed by novel jailbreak techniques. To address these limitations, we introduce a toxicity-aware distance metric that jointly incorporates toxicity scores and semantic similarity, overcoming the inadequacies of each measure in isolation. Building upon this metric, we propose a regression-based probabilistic certification framework for VLMs via randomized smoothing, providing formal guarantees of non-toxic output generation under both Gaussian (ℓ₂) and Laplacian (ℓ₁) perturbations. Distinct from prior randomized smoothing approaches that inject noise in the pixel space, our method injects noise directly into the visual embedding space, enabling robust certification against both adversarial and structure-based jailbreak attacks. Extensive experiments on MiniGPT-4, Qwen2-VL, and CogVLM validate the proposed framework across diverse model architectures, noise scales, and toxicity thresholds, consistently demonstrating strong empirical alignment with theoretical certification guarantees.
TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models
Published in arXiv preprint arXiv:2503.10872, 2025
Vision Language Models demonstrate strong inference abilities but face vulnerabilities to jailbreak attacks. We propose TAIJI, a black-box defense framework that employs key phrase-based textual anchoring to enhance the model’s ability to assess and mitigate harmful content embedded within both visual and textual prompts. Unlike existing methods that require model access or multiple queries, TAIJI operates with a single inference query while maintaining performance on legitimate tasks. Extensive evaluations show that TAIJI substantially improves VLM safety and reliability for real-world deployment.
Toward Linearly Regularizing the Geometric Bottleneck of Linear Generalized Attention
Published in Transactions on Machine Learning Research, 2025
Linear generalized attention mechanisms suffer from a geometric bottleneck that limits representational capacity and downstream robustness. This work provides a theoretical analysis of the bottleneck through the lens of information geometry and proposes linear regularization strategies to mitigate it. By introducing structured constraints on the attention feature maps, the approach improves both the expressiveness and stability of linear attention models, achieving consistent gains on language modeling and classification benchmarks while maintaining the computational efficiency that linear attention is designed to offer.
Cumulative Consensus Score: Label-Free and Model-Agnostic Evaluation of Object Detectors in Deployment
Published in arXiv preprint arXiv:2509.12871, 2025
We introduce the Cumulative Consensus Score (CCS), a method for evaluating object detection models without ground-truth annotations. CCS applies test-time data augmentation to each image and measures the spatial consistency of predicted bounding boxes across augmented views using Intersection over Union. The approach achieves over 90% congruence with F1-score, Probabilistic Detection Quality, and Optimal Correction Cost in controlled experiments. Operating in a model-agnostic manner across single-stage and two-stage detectors, CCS works at the case level to identify underperforming scenarios and provides a robust foundation for DevOps-style continuous monitoring of object detectors in real-world deployment.
GatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
Published in arXiv preprint arXiv:2512.07782, 2025
Modern autoregressive models rely on attention, yet the Softmax full attention in Transformers scales quadratically with sequence length. While Sliding Window Attention (SWA) improves efficiency, its difference-style update under an Associative Memory interpretation renders the training objective effectively unbounded. We propose GatedFWA, which preserves SWA’s efficiency while stabilizing memory updates and making gradient flow controllable through a learnable decay bias accumulated as per-token/head gates in attention logits. The implementation includes a fused one-pass gate preprocessing and a FlashAttention-compatible kernel. Experiments demonstrate competitive throughput with negligible overhead and better use of global context on language modeling tasks, with seamless integration with token compression methods.
FALCON: Fine-Grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model
Published in Advances in Neural Information Processing Systems (NeurIPS) 2025, 2025
Large language models have been widely applied but can inadvertently encode sensitive or harmful information, creating significant safety challenges. We propose FALCON, a machine unlearning approach that goes beyond coarse-grained loss combinations. FALCON employs three key mechanisms: information-theoretic guidance for selecting which parameters to modify, contrastive mechanisms to better separate learned representations, and orthogonal projection of conflicting gradients to balance forgetting objectives with model utility. Experiments demonstrate that FALCON achieves effective knowledge removal while preserving overall model performance and resisting knowledge recovery attempts.
Fragile by Design: On the Limits of Adversarial Defenses in Personalized DreamBooth Generation
Published in AAAI Conference on Artificial Intelligence 2026, 2026
This paper examines vulnerabilities in privacy-protecting mechanisms for customized image generation systems such as DreamBooth. We identify two significant weaknesses in existing defenses: adversarial examples often exhibit perceptible artifacts, and the perturbations can be eliminated through simple filtering. We introduce AntiDB_Purify, a testing framework that evaluates defenses against realistic purification attacks. Our findings demonstrate that current protective methods fail under such threats, revealing that they provide insufficient security for preserving user identity in personalized generation applications.
ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoning
Published in European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2026), 2026
Retrieval-Augmented Generation (RAG) improves the reliability of large language model applications by grounding generation in retrieved evidence, but it also introduces a new attack surface: corpus poisoning. We propose ProGRank, a defense mechanism that operates at the retriever level without requiring retraining. The approach stress-tests each query–passage pair under mild randomized perturbations and extracts probe gradients from the retriever’s parameters to identify potentially poisoned content. By combining representational consistency and dispersion risk metrics in a reranking step, ProGRank provides stronger defence performance and a favorable robustness–utility trade-off, while remaining effective against adaptive attacks across multiple datasets and retriever architectures.
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.