This final showcase shows everything working together, how it works, potential use cases, and future improvements and features that could be implemented in the project.
Members who worked on the project:
Fernando Rodriguez-Estrada
Ricardo Wu
Shengxuan (Shawn) Wang
Myself :)
Project Showcase
High-level overview diagram
Low-level diagram overview with future implementations included (from original concept design)
Source code
Ultraleap Desktop: https://github.com/OSU-App-Club/GestureProject-UltraLeap
NextJS web app reads hand gestures from Ultraleap 3Di camera and user interface is displayed through Holo Industries holographic plate
Gestures are sent to WebSocket proxy
WebSocket proxy forwards all messages via Kafka producer
AWS Kafka → RabbitMQ via gRPC server → client, respectively: https://github.com/OSU-App-Club/GestureProject-AWS
Deployed on AWS ECS cluster via AWS Copilot
Server service is a gRPC server that consumers Kafka messages and forwards them to any connected gRPC clients
Client service is a gRPC client that receives messages from the gRPC server and forwards them to the appropriate RabbitMQ queue based on the user pin code
Client Desktop: https://github.com/OSU-App-Club/GestureProject-ClientDesktop
RabbitMQ messages are consumed and sent to WebSocket proxy
WebSocket proxy notifies web client of messages
NextJS web app listens for messages and controls the mouse via the RobotJS NPM package
Comments