SINIC X research projects

Direct VLM-to-VLA Adaptation via Language-Action Grounding

CLAP (Causal Language-Action Prediction) turns pretrained VLMs into VLAs by prepending a natural-language action plan to each numeric action chunk, so action prediction is causally conditioned on it with no architectural modification, action expert, or extra annotation. 2B CLAP hits 90.8% on LIBERO (+14.9 pt over VLA-0) with 4-5 hours of fine-tuning.

Online Robot Manipulation Error Detection and Description

We formulate robot failure detection as online video temporal grounding, predicting the temporal boundaries and a cause-and-effect description of each manipulation failure as the video streams.

Robust Object Insertion via Masked Encoding and Soft Wrist

A soft wrist collects safe tactile demos, and masked trajectory encoding retrieves them to insert unseen pegs at 85% success.

Compliance-Enabled Tactile Phase Retrieval for Few-Shot Insertion Learning

A compliant wrist reveals insertion phases from touch, so phase-consistent retrieval lifts few-shot peg-in-hole success by 13 points.

A Slack-Compensating, Helical-Buckling-Attenuating Flexible-Shaft Transmission for Lightweight Multi-DoF Manipulation

This study proposes SHAFT, a flexible-shaft transmission mechanism that compensates for slack with a proximal tensioner to suppress helical buckling in lightweight, remotely actuated manipulators. Compared with a flexible shaft without a tensioner, SHAFT improves transmission efficiency by about 30% and reduces angular transmission error by about 65% on a path with four 90-degree bends. Using SHAFT, we built a 6-DoF arm with a 1-DoF gripper whose arm module weighs only 280 g.

A Layout-Aware Benchmark Reveals Memory Layout Matters for Graph-based ANNS on GPU

Decoupling graph topology from memory layout on GPU. Vertex reordering alone yields up to 80% QPS gain at equal recall.

Can Compact Language Models Search Like Agents? Distillation-Guided Policy Optimization for Preserving Agentic RAG Capabilities

We investigate whether compact language models (0.5–1B) can acquire sophisticated agentic retrieval-augmented generation (RAG) behavior. While existing agentic RAG systems rely on multi-billion-parameter models, compact models suffer from poor initial outputs, sparse rewards, and unstable reinforcement learning dynamics. To address these challenges, we propose Distillation-Guided Policy Optimization (DGPO), a two-phase training framework combining cold-start knowledge distillation and selective teacher-guided reinforcement learning.

Grounded Vision-Language Interpreter for Integrated Task and Motion Planning

While recent advances in vision-language models (VLMs) have accelerated the development of language-guided robot planners, their black-box nature often lacks safety guarantees and interpretability crucial for real-world deployment. This paper proposes ViLaIn-TAMP, a hybrid planning framework for enabling verifiable, interpretable, and autonomous robot behaviors.

A Flexible Wrist with Anisotropic and Switchable Stiffness for Contact-Rich Manipulation

CLAW is a lightweight, low-cost flexible wrist that improves learned contact-rich manipulation through passive 6-DoF compliance and switchable anisotropic stiffness.

Robust and Resilient Soft Robotic Object Insertion with Compliance-Enabled Contact Formation and Failure Recovery

By integrating VLM-guided failure recovery into compliance-enabled contact formation, we achieve robust and resilient soft robotic object insertion without force sensing or controller retraining.

Learning Diffusion Policies from Demonstrations For Compliant Contact-rich Manipulation

DIPCOM: Learning Diffusion Policies from Demonstrations For Compliant Contact-rich Manipulation. (Code to be released soon)

Soft Conical Universal Robotic Hand for Scooping Granular Media from Containers of Various Sizes

This study proposes SCU-Hand, a soft and reconfigurable end-effector designed to scoop powdered materials from containers of various sizes using a general-purpose robotic arm. By deforming a single thin sheet into a conical shape, the SCU-Hand achieves adaptive surface contact without complex sensors or control algorithms, while maintaining the stiffness needed for efficient scooping. Experimental results demonstrate that SCU-Hand achieves over 95% scooping performance across different container sizes and granular materials, outperforming conventional rigid tools by up to 20%.

Bridging the Gap between Scientific Papers and Poster Layouts

SciPostGen is a large-scale dataset of paper–poster pairs. It enables applications such as retrieving layouts aligned with a given paper and using them to guide poster layout generation.

Rethinking the role of frames for SE(3)-invariant crystal structure modeling

To make a GNN invariant to rotations, let's standardize the orientations of local atomic environments represented by internal self-attention weights, instead of directly standardizing the global structure.

Pre-training Figure QA from Stage-by-Stage Synthesized Images

SBS Figures: Pre-training Figure QA from Stage-by-Stage Synthesized Images

Infinitely Connected Attention for Periodic Structure Encoding

Propose a transformer for crystal property prediction by mimicking interatomic potential summations via self-attention.

Neural structure fields with application to crystal structure autoencoders

Express crystal structures as 3D continuous fields for crystal structure prediction

Interactive Optimization With LLM-Assisted Preference-Based Problem Instantiation

LAPPI uses LLMs to transform vague natural-language preferences into solver-ready optimization problems, enabling an interactive loop for tasks such as trip planning.

Cooperative Design Optimization through Natural Language Interaction

In this work, we propose the concept of cooperative design optimization through natural language. To realize this, we present a novel technique to integrate an LLM with the Bayesian Optimization (BO)-based optimization procedure, enabling designers to intervene through natural language and understand the system’s intentions behind its suggestions.

What Novice Operators See Changes What the Policy Learns: A User Study of Keypoint Hand Visual Representation for Teleoperated Demonstration

What novices see shapes the demonstrations they collect, so the operator's view and the policy's observation are worth designing together.

FairTalk - Facilitating Balanced Participation in Video Conferencing by Implicit Visualization of Predicted Turn-Grabbing Intention

Toward AI-Mediated Avatar-Based Telecommunication: Investigating Visual Impression of Switching Between User- and AI-Controlled Avatars in Video Chat

Our target scenario is a video chat between two people using avatars, where one person uses an auto-switching mechanism to switch between user-controlled and AI-controlled modes. Our study shows that appropriate auto-switching improves the user experience, highlighting the importance of future research in advancing this auto-switching mechanism.