Webcam Eye Tracker: Eye Tracking Video Games

Now that we have a working predictive model, we can deploy it to a simple application to test how well the eye tracker works. The plan is three-fold: Create a Predictor class that can load a trained model and make predictionsAdd a "tracking" mode to the data collector as a quick way to test the PredictorCreate a simple screen recorder that can save videos of eye tracking while playing video games Predictor class We want to create a predictor class that can handle all of the model loading and predicting. PyTorch models can be saved either as checkpoint files or…

0 Comments

Webcam Eye Tracker: Deep Learning with PyTorch

So far we have extracted webcam features and collected coordinate data. Now we can use that dataset to create our deep learning model with PyTorch. The following models and analyses were conducted in a Jupyter notebook, which can be found here. The problem we have is basically bounding box regression, but simplified to only 2 continuous output values (X-Y screen coordinate). To summarize, the data we have available to us: Possible inputsUnaligned face (3D Image)Aligned face (3D Image)Left eye (3D Image)Right eye (3D Image)Head position (2D Image)Head angle (Scalar)OutputsX screen coordinateY screen coordinate The goal is to find the most…

0 Comments