AI / ML · Software · Robotics

Somak Shivlani

Building intelligent systems across AI, robotics, and infrastructure

ProjectsResumeGitHubLinkedInEmail

About

Who I am

I'm an AI/ML and software engineer focused on systems that actually ship — from privacy-first personal intelligence to research on how we evaluate deep models. I care about the full stack: data, models, infrastructure, and the product surface that makes the work useful.

My background spans machine learning engineering, computer vision, robotics coursework, and startup building. I'm currently founding ML at Textile, pursuing an M.S. in Computer Science at the University of Miami, and continuing research on parameter-efficient adaptation for 3D medical visual question answering.

Selected work

Projects with technical depth

A short set of systems I've built or contributed to — framed by problem, approach, and outcome rather than a tech checklist.

01

Textile — Private On-Device AI

Founding Lead ML Engineer building a privacy-first personal intelligence layer that searches and reasons over local documents without sending data to the cloud.

  • LLMs
  • RAG
  • Embeddings
  • OCR
  • On-device ML
  • Python
textile.dev →
Problem
People need AI that understands their own files, but most assistants require uploading sensitive documents to third-party servers.
What I built
Core ML systems for on-device document understanding, semantic search, and contextual question answering across PDFs, Office files, and images.
Approach
Local embeddings, retrieval pipelines, OCR/document parsing, and streaming inference designed for privacy, latency, and large personal corpora.
Result
Shipped product foundations for Textile — private search and detailed answers where nothing leaves the device.
02

AI Support Agent

TypeScript support agent that retrieves knowledge-base articles, runs multi-turn grounded troubleshooting, cites sources, and escalates into structured cases when docs are exhausted.

  • Next.js
  • TypeScript
  • RAG
  • Gemini
  • Zod
  • Embeddings
Problem
Support bots often invent fixes or dump walls of text instead of grounded, step-by-step diagnosis with clear handoff.
What I built
A Next.js chat agent with semantic retrieval over a Markdown KB, Zod-validated agent decisions, citations, and simulated case creation.
Approach
Gemini embeddings with cosine retrieval and lexical fallback, structured prompts, one-question-at-a-time diagnostics, and separate prod vs inspector UIs.
Result
Demo-ready system that can resolve documented issues or escalate cleanly with a machine-readable case summary.
03

ControlNet LoRA for Chart Generation

Computer vision final project adapting Stable Diffusion + ControlNet with LoRA to generate charts conditioned on layout structure, evaluated with OCR and layout IoU.

  • Stable Diffusion
  • ControlNet
  • LoRA
  • PyTorch
  • Diffusers
  • OCR eval
Problem
Generic diffusion models struggle to produce charts with reliable structure, labels, and layout fidelity.
What I built
Synthetic line-chart dataset generation, ControlNet LoRA training, baseline comparisons, and quantitative evaluation scripts.
Approach
Condition on layout/segmentation signals, fine-tune with LoRA for lightweight adaptation, and score outputs via OCR + layout overlap metrics.
Result
A reproducible pipeline for controllable chart generation with measurable layout quality against baselines.
04

RL Grasping on HSRB

ROS package for reinforcement learning of tabletop grasping on Toyota’s HSRB, spanning discrete arm/gripper actions, reward shaping, and Isaac Sim integration.

  • ROS
  • Reinforcement Learning
  • Isaac Sim
  • Python
  • HSRB
Problem
Hand-crafted grasp heuristics break down as object pose and workspace geometry vary — the arm needs a learned closed-loop policy.
What I built
An HSR grasp environment node with discretized state, Q-learning style training loops, joint trajectory control, and launch/demo scripts for Omniverse/Isaac Sim.
Approach
Sparse-to-shaped rewards around end-effector proximity and lift success, epsilon-greedy exploration, and ROS messaging for proprioception and control.
Result
A working research prototype for learning grasp behaviors on a real robot platform in simulation.
05

RTWebGL — Path Tracing in the Browser

A WebGL fragment-shader path tracer with reflective spheres, a Cornell-box style room, and interactive bounce controls — runnable without a dedicated GPU.

  • WebGL
  • GLSL
  • Ray Tracing
  • JavaScript
Open live demo →
Problem
Real-time ray tracing demos usually require native engines or heavy desktop GPUs, which makes the ideas hard to share.
What I built
A self-contained WebGL renderer that traces rays on the GPU, accumulates reflective bounces, and exposes bounce count + FPS controls.
Approach
GLSL ray-sphere/plane intersection, recursive reflection up to a configurable bounce depth, and a lightweight JS bootstrap around classic WebGL utilities.
Result
An interactive demo you can open in the browser — now embedded on this site.

Experience

Where I've built

  1. Feb 2025 — Present

    Founding Lead Machine Learning Engineer

    Textile Labs · Miami, FL

    • Co-founded a privacy-first AI startup and led development of a distributed on-prem platform for document indexing, API orchestration, OCR processing, and large-scale enterprise data retrieval.
    • Designed scalable backend services for stateful workflow execution, integrating document processing, email automation, and structured data APIs into a unified platform.
    • Engineered orchestration pipelines across 6+ services with structured routing, validation, asynchronous task execution, and fault-tolerant error handling.

  2. Jan 2025 — Dec 2025

    Engineering Data Scientist Intern

    Cummins Cederberg Coastal & Marine Engineering · Coral Gables, FL

    • Built a large-scale document indexing and retrieval platform with FAISS and local inference that expanded searchable internal documentation coverage by 80%.
    • Deployed an on-prem LLM assistant with Docker, llama.cpp, and FastAPI for 50+ engineering users across 3 departments, with structured prompting and output validation.
    • Built a computer vision pipeline to detect boat-engine scarring in seagrass from satellite imagery, reaching 85% precision across 3,000 annotated image samples.

  3. Aug 2024 — May 2026

    Research Assistant

    University of Miami, Department of Computer Science · Coral Gables, FL

    • Developed PyTorch-based semi-automatic segmentation pipelines that reduced manual labeling requirements across image annotation workflows.
    • Trained and evaluated ResNet18-based models across 20 experiments using Dice score, cosine similarity, and angular similarity to compare segmentation performance.

  4. Aug 2024 — May 2026

    Project Manager

    Bonsai Applied Computations Group · Coral Gables, FL

Research & education

Study & inquiry

M.S. in Computer Science

University of Miami · August 2026

B.S. in Computer Science, Minor in Mathematics. May 2026

Interests

  • Large Language Models
  • Multimodal Large Language Models
  • Vision Transformers
  • Parameter-efficient fine-tuning
  • 3D image processing
  • Deep learning
  • AI/ML systems
  • Computer vision
  • Autonomous robotics

Selected coursework

  • Machine Learning
  • Statistical Learning
  • Computer Vision
  • Autonomous Robotic Systems
  • Computer Graphics
  • Software Engineering
  • Cybersecurity

Master's thesis

Parameter-Efficient Adaptation to M3D Models for Med-VQA

University of Miami · M.S. in Computer Science · August 2026

Supervised by Professor Dilip Sarkar

This thesis studies parameter-efficient adaptation for three-dimensional medical visual question answering. Starting from frozen M3D-LaMed backbones, it compares a visual-token Performance-Enhancing Multimodal Adapter (PEMA) against rank-8 LoRA and published references on closed-ended 3D-RAD Tasks 4–6. PEMA adapts at the projected visual-token interface while keeping the vision backbone and language model frozen, recovering substantial task performance while training less than 0.31% of combined model parameters — with diagnostics showing when aggregate accuracy is driven by answer priors versus image-sensitive behavior.

  • 3D radiology
  • Med-VQA
  • M3D-LaMed
  • PEFT
  • PEMA
  • LoRA
  • Visual-token adaptation

Contact

Want to build something interesting?

Open to conversations about AI systems, research collaborations, and ambitious product work.

somakshivlani@gmail.comLinkedInGitHub