Encoding Domain Insights into Multi-modal Fusion: Improved Performance at the Cost of Robustness

Michaels, J. S., Zhang, S., & Fiterau, M.

MOSS Workshop @ ICML 2025

Challenges in Understanding Modality Conflict in Vision-Language Models

Nguyen, T*., Michaels, J.* S., Fiterau, M., & Jensen, D.

Actionable Interpretability Workshop @ ICML 2025

Audio-Journey: Open Domain Latent Diffusion Based Text-to-Audio Generation

Michaels, J.*, Li, J. B.*, Yao, L., Yu, L., Wood-Doughty, Z., & Metze, F.

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Seoul, Korea, 2024

Audio-Journey: Efficient Visual+LLM-aided Audio Encodec Diffusion

Li, J. B.*, Michaels, J.*, Yao, L., Yu, L., Wood-Doughty, Z., & Metze, F.

Workshop on Efficient Systems for Foundation Models @ ICML 2023

Kinematic imaging of salticid eye tubes with noisy X-ray video

Michaels, J., Korman, D., Harris, O., Alexander, E., & Morehouse, N.

Presented at Sensorium Conference, 2023

Adapting Latent Diffusion Models from Image to Class-Conditioned Audio Generation

Michaels, J.

MS Thesis, Department of Computer Science, Northwestern University, Evanston, IL, USA, 2023

Progress and push-back: How the killings of Ahmaud Arbery, Breonna Taylor, and George Floyd impacted public discourse on race and racism on Twitter

Nguyen, T., Criss, S., Michaels, E., Cross, R., Michaels, J., Dwivedi, P., Huang, D., Hsu, E., Mukhija, K., Nguyen, L., Yardi, I., Allen, A., Nguyen, Q., & Gee, G.

SSM - Population Health, 15100922

Graduate Researcher

University of Massachusetts, Amherst, MA | Advisor: Prof. Ina Fiterau

September 2023 - Present

• • Investigating the robustness of multi-modal models against targeted and random adversarial attacks to identify key architectural vulnerabilities.
• • Developing novel interpretability methods to analyze how Vision-Language Models (VLMs) behave when presented with conflicting information across modalities.
• • Designing experiments to probe the failure modes of multi-modal systems, aiming to improve model reliability and safety.

Score Based Audio Generation

Northwestern University, Evanston, IL | Advisor: Prof Zach Wood-Doughty

• • Completed Master's Thesis on adapting latent diffusion models from image to audio domain with novel auto-encoder / diffusion model architecture
• • Improved downstream classifier accuracy through large scale data augmentation with generated samples
• • Designed and implemented multiple custom diffusion models and training schemes for task specific use cases

Arachnid Eye Tracking

Northwestern University, Evanston, IL | Advisor: Prof Emma Alexander

• • Collaboration between the Bio Inspired Vision Lab at Northwestern and the Morehouse Lab at the University of Cincinnati
• • Experimented with cutting edge image processing techniques for denoising and feature extraction
• • Implemented pipeline to extract low visibility features from X-Ray data
• • Trained multiple models of extracted features for tracking
• • Designed and implemented filtering for noisy tracking results to allow scientific insights from noisy data

Twitter Data Analysis for Health Equity

Big Data 4Health Equity

• • Handled Twitter API interaction to extract a large dataset of twitter data along with sentiment analysis
• • Developed tools to cross reference twitter data with search data retrieved from google analytics to expand the relevant dataset
• • Aided in developing data visualizations and analysis for online behavior tracking

Data Science Hub Research

Champlain College, Burlington, VT | Advisor: Prof Narine Hall

• • Conducted a literature review on Self-Organizing Maps
• • Designed and developed website for Champlain College Data Science Hub

Graduate Application Mentor

Fall 2022

• • Mentored two students from underrepresented backgrounds through the competitive PhD application process as part of the Application Help initiative
• • Provided comprehensive, multi-session guidance on application strategy, from school selection to effective faculty outreach
• • Conducted detailed reviews and provided actionable feedback on critical application materials, including personal statements, statements of purpose, and CVs

GraduateStudent Ambassador

Spring 2023

• • Volunteered for the Candidate Friday admitted student event
• • Engaged with candidates to provide authentic insights into the program curriculum, research opportunities, and student life at Northwestern