Samsung Galaxy Smartwatch As A Game Controller

This post is part of the Into The Past series. A list of posts written long after they happened as part of an attempt to look back at some earlier creations and thoughts I had along the way. This post was written May 2024.

Introduction #

I had moved to Vancouver, Canada for a few months right in the midst of the Covid19 outbreak. Looking past the worries and struggles that we all faced during that time, I want to highlight a small demo that I was working on during my stay at the Centre For Digital Media.

I was working a project on a small team in collaboration with the Samsung R&D Institute Canada, in order to design and implement a proof of concept for transfering and controlling files between multiple samsung devices. We were working with interconnectivity between their smartwatches and their smart TVs.

Knowing that their watches came with an in-built gyroscope, I figured that it would a be a fun experiment to see if we can extend our wireless communication to also have real-time game controls.

Demonstration #

Without adding too many details in advance, let's have a look at the prototype in action running on a Samsung Galaxy Watch Active2:

Details #

The prototype host is a simple breakout game running on a separate machine as a web application that listens for incoming websocket connections from the smartwatch. Once a websocket connection has been established, the game will start and listen for events fired from the watch.

I was utilizing the Gyroscope rotation vector sensor API to validate whether or not the watch has been rotated relative to the arms forward facing direction. However, as we can see from the video, the registration is shaky at best.

I have applied a small cutoff-value that will ignore any relative rotation less than 10% to avoid the noise of shaky arms and the internal gyroscope moving around. This value can be tweaked a bit for better registration. Additionally, there is a 100ms minimun time passing between 2 consecutive events being fired. This can also be reduced to avoid the delay that we see from rotating the arm to the game pad moving either left or right.

One limitation that I was facing is the fact that the arm needs to be stretched horisontally. One solution for more precise rotation registration as well as removing the stretched arm limitation, would be to calculate the relative rotation using the gyroscope orientation sensor vector and offset this by the watches current position in space / watchface direction from the in build accelerometer. This however was never investigated further during building of this prototype.

Below is a stripped down implementation of the functionality we see in the video:

var tizen = ... // Tizen framework refernece
var socket = ... // instance of socket connected to the web based game
var gyroscopeCapability = tizen.systeminfo.getCapability('http://tizen.org/feature/sensor.gyroscope_rotation_vector');
var minimumGyroscopeReadDelay = 100;
var currentDirection = 0;

if (gyroscopeCapability === true) {
    var gyroscopeSensor = tizen.sensorservice.getDefaultSensor('GYROSCOPE_ROTATION_VECTOR');
    gyroscopeSensor.start(onGyroscopeSensorStarted);
    gyroscopeSensor.setChangeListener(onGyroscopeSensorValueChanged, minimumGyroscopeReadDelay, 0);
}

function onGyroscopeSensorValueChanged(sensorData) {
    var noiseCutoffValue = 0.1;
    var directionChanged = false;

    if (Math.abs(sensorData.y) > noiseCutoffValue) {
        if (sensorData.y > 0 && currentDirection != -1) {
            // moving left
            currentDirection = -1;
            directionChanged = true;
        } else if (sensorData.y < 0 && currentDirection != 1) {
            // moving right
            currentDirection = 1;
            directionChanged = true;
        }
    } else if (currentDirection != 0) {
        currentDirection = 0;
        directionChanged = true;
    }

    if (directionChanged) {
        socket.emit('drawing', {
            direction: currentDirection;
        });
    }
}