@@ -45,24 +45,24 @@ Sleep is an essential activity we ideally spend on every night.
...
@@ -45,24 +45,24 @@ Sleep is an essential activity we ideally spend on every night.
Every time one sleeps, one cycles through 3 main stages of sleep (light, deep, and rapid-eye movement (REM)), each of which contributes to maintaining their physical and mental health.
Every time one sleeps, one cycles through 3 main stages of sleep (light, deep, and rapid-eye movement (REM)), each of which contributes to maintaining their physical and mental health.
However, with the multitude of attention and stresses that occur in modern life, it can be hard to get adequate sleep.
However, with the multitude of attention and stresses that occur in modern life, it can be hard to get adequate sleep.
This can lead to a build up of sleep debt, which, as stated by~\cite{luyster2012sleep}, significantly impairs cognitive and motor functions, as well as increasing disease and accident risk, decreasing longevity, and increased individual and societal costs (e.g., healthcare).
This can lead to a build up of sleep debt, which, as stated by~\cite{luyster2012sleep}, significantly impairs cognitive and motor functions, as well as increasing disease and accident risk, decreasing longevity, and increased individual and societal costs (e.g., healthcare).
There are many avenues for tackling the problem of sleep deprivation through technology, from habit development apps to blocking timeout that stops the user from using their smart devices after some time.
There are many avenues for tackling the problem of sleep deprivation through technology, from habit development apps to blocking app that stops the user from using their smart devices after a set time.
The pathway I choose to focus on is augmenting the capabilities of alarm clocks.
The pathway I choose to focus on is augmenting the capabilities of alarm clocks.
The alarm clock itself is a simple device that is designed to wake up the user at a set time.
The alarm clock itself is a simple device that is designed to wake up the user at a set time.
Within this simplicity there is a problem though, which is that alarm clocks have no idea what stage of sleep the user is in when the alarm goes off.
Within this simplicity there is a problem though, which is that alarm clocks have no idea what stage of sleep the user is in when the alarm goes off.
This can cause the alarm to go off during inopportune times, which can lead to the user feeling groggy, snoozing the alarm, or at worst, not even waking up.
This can cause the alarm to go off during inopportune times, which can lead to the user feeling groggy, snoozing the alarm, or at worst, not even waking up.
A way to fix this issue would be to have the alarm clock contain a mechanism for determining where along the sleep cycle the user is.
A way to fix this issue would be to have the alarm clock contain a mechanism for determining where user is along the sleep cycle.
This can allow the alarm to more gently wake the user up and can help them rely less on snoozing the alarm.
This can allow the alarm to more gently wake the user up and can help them rely less on snoozing the alarm.
Many people have fitness devices that are able to measure sleep such as a Fitbit watches, a SleepScore on-table devices, or a Withing under-mattress devices.
Many people have fitness devices that are able to measure sleep such as Fitbit watch, a SleepScore on-table device, or a Withing under-mattress device.
According to~\cite{10.1093/sleep/zsaa291}, consumer level sleep tracking on average do very well with detecting wakefulness and sleep, and can sometime achieve results as good as professional devices.
According to~\cite{10.1093/sleep/zsaa291}, consumer level sleep trackers on average do very well with detecting wakefulness and sleep, and can sometime achieve results as good as professional devices.
However, they are mainly able to achieve this with consistent sleep results based on user information, and may not be as accurate for nights with poor or abnormal sleep.
However, they are mainly able to achieve this with consistent sleep results based on user information, and may not be as accurate for nights with poor or abnormal sleep.
Even so, the relative information provided by these devices can be used to tailor models to the individual's sleep patterns.
Even so, the relative information provided by these devices can be used to tailor models to the individual's sleep patterns.
There have been application that have applied similar the one I am describing, namely Sleep Cycle for iOS and Android.
There have been applications that have applied similar the one I am describing, namely Sleep Cycle for iOS and Android.
The way the Sleep Cycle application works is by having the user set a time window range to wake up, rather than just a single time point.
The way the Sleep Cycle application works is by having the user set a time window range to wake up, rather than just a single time point.
The application will then guess based on sound and movement approximately what sleep stage the user is in and then try to wake up the user at the best possible time within that time window.
The application will then guess based on sound and movement approximately what sleep stage the user is in and then try to wake up the user at the best possible time within that time window.
My application will take this idea and focus more on bringing this functionality to non-smartphone devices by using the Withing Sleep Analyzer and Raspberry Pi.
My application will take this idea and focus more on bringing this functionality to non-smartphone devices by using the Withing Sleep Analyzer.
\section{Technical Approach}\label{sec:technical}
\section{Technical Approach}\label{sec:technical}
The overall approach to the project can be split into two primary parts: the alarm clock and the AI model.
The overall approach to the project can be split into two primary parts: the alarm clock and the AI model.
...
@@ -72,7 +72,7 @@ In addition, I will also discuss how the data used for model training.
...
@@ -72,7 +72,7 @@ In addition, I will also discuss how the data used for model training.
For the data in this project, I used sleep data provided to me by a participant who uses the Withing Sleep Mat.
For the data in this project, I used sleep data provided to me by a participant who uses the Withing Sleep Mat.
The Withing Sleep Mat Analyzer, is an under-mattress device that takes measurements using air pressure to measure user movement (such as respiration) and electronic sensor to measure heart rate.
The Withing Sleep Mat Analyzer, is an under-mattress device that takes measurements using air pressure to measure user movement (such as respiration) and electronic sensor to measure heart rate.
This is then used to to calcualte the person's sleep time, sleep efficiency, and approximate sleep stages.
This is then used to to calcualte the person's sleep time, sleep efficiency, and approximate sleep stages.
According to~\cite{Edouard2021}, does very well with time to sleep against a traditional polygraphy device, which is useful for the project as this is the primary method used to determine when to start the alarm.
According to~\cite{Edouard2021}, the Withing Sleep Analyzer does very well with time to sleep against a traditional polygraphy device, which is useful for the project as this is the primary method used to determine when to start the alarm.
I extracted the data as a CSV file from the Withing website, and it contained slightly over 3 years of data, with row in the data specifying a sleep event (in UTC time), the stages determined, and the duration.
I extracted the data as a CSV file from the Withing website, and it contained slightly over 3 years of data, with row in the data specifying a sleep event (in UTC time), the stages determined, and the duration.
According to~\cite{strathprints79694}, one needs to have over 2 months of sleep data to account for variability in sleep (such as due to daylight saving time, seasonality, and outlier periods of sleep), so having this much data is very useful for determining the user's consistent sleep pattern.
According to~\cite{strathprints79694}, one needs to have over 2 months of sleep data to account for variability in sleep (such as due to daylight saving time, seasonality, and outlier periods of sleep), so having this much data is very useful for determining the user's consistent sleep pattern.
There was also extra data on heart rate and respiration for the sleep time, however I did not use them for this project.
There was also extra data on heart rate and respiration for the sleep time, however I did not use them for this project.
...
@@ -104,7 +104,7 @@ The transition between the states would be:
...
@@ -104,7 +104,7 @@ The transition between the states would be:
Using this structure allowed me to modularize the code for the alarm clock and easily slot extra elements such as audio using a VLC instance and the AI model.
Using this structure allowed me to modularize the code for the alarm clock and easily slot extra elements such as audio using a VLC instance and the AI model.
\subsection{AI Model}\label{sec:model}
\subsection{AI Model}\label{sec:model}
During the consideration process, I looked at Feed-Forward Networks, Convolution Neural Networks (CNNs), and Recurrent Neural Networks (RNNs), as well as some non-neural network models such as Markov Chain.
During the consideration process, I looked at Feed-Forward Networks, Convolution Neural Networks (CNNs), and Recurrent Neural Networks (RNNs), as well as some non-neural network models such as Markov Chains.
Each of the models considered required taking a set range of sleep data (with prior probabilities), running calculations, and then return new probabilities for the next time step.
Each of the models considered required taking a set range of sleep data (with prior probabilities), running calculations, and then return new probabilities for the next time step.
\begin{figure}[h]
\begin{figure}[h]
\centering
\centering
...
@@ -165,7 +165,7 @@ Moving into the smart\_alarm\_clock.py script, the code structure for is kept mo
...
@@ -165,7 +165,7 @@ Moving into the smart\_alarm\_clock.py script, the code structure for is kept mo
The three major additions for this script are:
The three major additions for this script are:
\begin{enumerate}
\begin{enumerate}
\item a function for loading and running the model (includes the softmax and model prediction helper functions),
\item a function for loading and running the model (includes the softmax and model prediction helper functions),
\item a deque (Double-Ended Queue) to represent the model's memory,
\item a deque (Double-Ended Queue) to represent the model's memory (set to a max-len of 5 [in minutes]),
\item and a VLC instance for playing sound from the script (this allowed it to sound like a regular alarm clock).
\item and a VLC instance for playing sound from the script (this allowed it to sound like a regular alarm clock).
\end{enumerate}
\end{enumerate}
...
@@ -174,12 +174,14 @@ The three major additions for this script are:
...
@@ -174,12 +174,14 @@ The three major additions for this script are:
For implementing the models, I first started by experimenting in a Jupyter Notebook to create the models and play around with setting up the parameters.
For implementing the models, I first started by experimenting in a Jupyter Notebook to create the models and play around with setting up the parameters.
Since each model had only a single hidden layer, the primary hyperparameter to tune for all the models was the number of units in the hidden layer.
Since each model had only a single hidden layer, the primary hyperparameter to tune for all the models was the number of units in the hidden layer.
To control for variations in the data, especially the randomization of the input probabilities, I made sure to save the data I generated into a separate CSV file and load the same information for each model.
To control for variations in the data, especially the randomization of the input probabilities, I made sure to save the data I generated into a separate CSV file and load the same information for each model.
The input layer is a batched 5 by 8 matrix representing 5 last known time step and their associated 4-stages (awake, REM, light sleep, and deep sleep) probabilities.
The output layer contains 4 output units for each stage's probability.
These are the specific models, defined with layer(units), I trained with their final parameter amounts:
These are the specific models, defined with layer(units), I trained with their final parameter amounts:
\item Input(8) $=>$ LSTM(16) $=>$ Dense(4): 1604 training parameters,
\item Input(5,8) $=>$ LSTM(16) $=>$ Dense(4): 1604 training parameters,
\item Input(8) $=>$ GRU(16) $=>$ Dense(4): 1268 training parameters,
\item Input(5,8) $=>$ GRU(16) $=>$ Dense(4): 1268 training parameters,
\item Input(8) $=>$ Attention(32) $=>$ Dense(4): 629 training parameters.
\item Input(5,8) $=>$ Attention(32) $=>$ Dense(4): 629 training parameters.
\end{itemize}
\end{itemize}
Once I finished experimenting with each of the models, I pushed them into their own Python script to allow training to be more parallelized.
Once I finished experimenting with each of the models, I pushed them into their own Python script to allow training to be more parallelized.
...
@@ -221,7 +223,7 @@ Since this is a time series data, I made sure that any time window that occurred
...
@@ -221,7 +223,7 @@ Since this is a time series data, I made sure that any time window that occurred
This made sure the models got an accurate subset of the data without missing pieces.
This made sure the models got an accurate subset of the data without missing pieces.
During training, all the models to initially perform too well, as every model, including the baseline, reached over 95\% accuracy.
During training, all the models to initially perform too well, as every model, including the baseline, reached over 95\% accuracy.
This is likely due to the very simple nature of the current data, as it only has 8 inputs, with only 4 of them being probabilities.
This is likely due to the very simple nature of the current data, as it only has 8 inputs per time step, with only 4 of them being probabilities.
This also lead the models to overfit with it favoring being awake over all the other sleep stages.
This also lead the models to overfit with it favoring being awake over all the other sleep stages.
I switch from categorical accuracy to categorical cross-entropy, which forced the model to take more clear stances on which sleep stage the user is in as it measures the residual between predicted and target, rather than how many it got correct.
I switch from categorical accuracy to categorical cross-entropy, which forced the model to take more clear stances on which sleep stage the user is in as it measures the residual between predicted and target, rather than how many it got correct.
This still had some overfitting issues, but was able to have more stable probabilities, especially within data in the middle of the night.
This still had some overfitting issues, but was able to have more stable probabilities, especially within data in the middle of the night.
...
@@ -269,7 +271,7 @@ The tools and scripts are there for it to work, with the current issue just bein
...
@@ -269,7 +271,7 @@ The tools and scripts are there for it to work, with the current issue just bein
The next extension on that would be to connect up the Withing API using Withing Python.
The next extension on that would be to connect up the Withing API using Withing Python.
While there isn't a way to stream sleep data as it is being collected, I can still use it to gather the sleep data after the user wakes up, which can allow the model to be trained on new data daily.
While there isn't a way to stream sleep data as it is being collected, I can still use it to gather the sleep data after the user wakes up, which can allow the model to be trained on new data daily.
Additionally, I can also hook into the notification system provided by the API for user in-bed and user out-of-bed to automate starting and stopping the alarm.
Additionally, I can also hook into the notification system provided by the API for user in-bed and user out-of-bed to automate starting and stopping the alarm.
Extending to a more general idea, there might be ways to add in other devices to provide sleep data tracking, such as FitBit or Sleep Cycle, to provide more supported devices for evaluation user sleep cycle.
Extending to a more general idea, there might be ways to add in other devices to provide sleep data tracking, such as FitBit or Sleep Cycle, to provide more supported devices for evaluation user sleep data.
In all, this project has hopefully shown that an offline and smartphone-less can be useful endeavor for improving alarm clocks.
In all, this project has hopefully shown that an offline and smartphone-less can be useful endeavor for improving alarm clocks.
