Blog

Machine Learning

Webcam Eye Tracker: Eye Tracking Video Games

Now that we have a working predictive model, we can deploy it to a simple application to test how well the eye tracker works. The plan is three-fold: Create a Predictor class that can load a trained model and make predictionsAdd a "tracking" mode to the data collector as a quick way to test the PredictorCreate a simple screen recorder that can save videos of eye tracking while playing video games Predictor class We want to create a predictor class that can handle all of the model loading and predicting. PyTorch models can be saved either as checkpoint files or…

0 Comments
January 4, 2021
Machine Learning

Webcam Eye Tracker: Deep Learning with PyTorch

So far we have extracted webcam features and collected coordinate data. Now we can use that dataset to create our deep learning model with PyTorch. The following models and analyses were conducted in a Jupyter notebook, which can be found here. The problem we have is basically bounding box regression, but simplified to only 2 continuous output values (X-Y screen coordinate). To summarize, the data we have available to us: Possible inputsUnaligned face (3D Image)Aligned face (3D Image)Left eye (3D Image)Right eye (3D Image)Head position (2D Image)Head angle (Scalar)OutputsX screen coordinateY screen coordinate The goal is to find the most…

0 Comments
December 21, 2020
Machine Learning

Webcam Eye Tracker: Data Collection of Screen Coordinates

In the previous post we extracted a number of facial features from our webcam image. Now, we need a way to get the "labels". So, we need to develop a semi-efficient data collection method to get our X-Y coordinate data. As this project is a learning project at its core, I wanted to come up with a way to collect enough data for the model without having to spend too much time doing it, and without having to manually label anything. We also can't collect too much redundant/repeated data because that will inflate our dataset size and make training take…

0 Comments
December 14, 2020
Machine Learning

Webcam Eye Tracker: Webcam Features and Face Detection

Now that we have a general overview of the project, the first step in creating our eye tracker is getting video from the webcam. Following that, we need to perform face detection, alignment, and calculate various features from that video stream. Webcam video We'll start by creating a Detector that uses OpenCV to retrieve frames from the webcam. By itself, reading from the webcam is quite straightforward: import cv2 capture = cv2.VideoCapture(0) capture.read() However, reading webcam frames is a blocking action and can cause quite a lot of slowdown in our application, so ideally we would do that retrieval in…

0 Comments
December 7, 2020
Machine Learning

Webcam Eye Tracker: An End-to-end Deep Learning Project

Recently, I wanted to learn PyTorch and needed to find a project to help focus my learning. I have always been interested in the idea of creating a webcam eye tracker, so that seemed like a good project for this. Eye trackers typically rely on infrared for accurate tracking, but performing the same task using purely vision techniques seemed like an interesting challenge. What follows is a series of posts on the process of creating a webcam eye tracker from scratch. As always, we should start by clarifying the main problems we're trying to address by going through this process.…

0 Comments
November 30, 2020
Gaming

Rise of Kingdoms: Talent Builder

We all know it's difficult to play around with talents in Rise of Kingdoms (RoK) because of the gem cost for talent resets. So, I spent the last little while working on an online Rise of Kingdoms talent builder where you can create, tweak, and share talent builds without worrying about talent resets. Here is the link to the Rise of Kingdoms talent builder: roktalents.com Note: The site is currently best viewed on PC or laptop. Functionality is limited on mobile/tablet and more work needs to be done to optimize the experience for mobile devices. Check your stats One feature…

5 Comments
February 19, 2020
Gaming

Rise of Kingdoms: Equipment Calculator

I have a long history of playing RPG games, where min-maxing and character planning were a huge part of the appeal. With the recent release of the equipment system in Rise of Kingdoms, I figured it might be useful to create an equipment calculator/planner. The goal was to create a system where you could mix and match equipment, with the total stat values displayed on the side. This would hopefully make it easier to fine tune your overall set to fit your play style. I threw together a rough version using Google Sheets. A standalone web app would probably be…

1 Comment
August 31, 2019
Gaming

Rise of Kingdoms: Saving gold keys

Here it is. The answer to the age-old question. The question that shows up at least once a day on Reddit and Discord: "should I save my gold keys and open them all at once?". The assumption being that opening gold keys together in a batch gives more/better rewards than opening them one at a time. Therefore, in this post we will test whether that is true. As always, you can find the code and dataset for this analysis in my GitHub repo here. Samples and expected distributions We'll begin with a quick primer on expected/population distributions and how they…

18 Comments
June 14, 2019
Gaming

Rise of Kingdoms: Show Your Love Event

In a previous post I discussed the point requirements of the last holiday event (Valentine's Day - Pledge of Thorns). Here, I will do the same analysis with the new holiday event (Show Your Love) to confirm those findings. I'll be drawing comparisons between the two events throughout. The supporting code and data for this analysis can be found here. Data collection for event I began by using 800 items (Carnations) and recorded how many points each item provided. Below (left) you can see the frequency distribution of the points provided for each item, as well as the distribution from…

2 Comments
May 10, 2019
Gaming

Rise of Kingdoms: Master’s Blueprints

In order to unlock Tier 5 research in Rise of Kingdoms you need a level 25 Academy. However, the upgrade to Academy level 25 has a number of nested building requirements, with the majority of those buildings needing a Master's Blueprint. So, what is the minimum number of Master's Blueprints required to unlock Tier 5 research? Here are the building requirements for a level 25 Academy and unlocking Tier 5: City hallAcademyTrading postGoldmineScout camp (no blueprint)Watchtower (no blueprint)WallTavernQuarryStorehouseHospitalCastle (no blueprint)Alliance centerSiege workshopBarracksFarmArchery rangeLumber MillStable(requires a Quarry but is already listed above) At minimum, you will need 16 Master's Blueprints, which…

0 Comments
April 29, 2019