Final Report

Smartfin Final Report

Jasmine Simmons, Brendon Ho,

Howard Wang, Samprith Kalakata

CSE 145/CSE 237D: Embedded Systems Design

University of California, San Diego

Spring 2019

Abstract

It is challenging for scientists to collect oceanographic data in nearshore environments because high energy wave dynamics make it difficult to deploy autonomous sensors. To address this, scientists at Scripps Institute of Oceanography have developed the Smartfin, a surfboard fin with embedded sensors. However, collecting oceanic data with the Smartfin poses new problems, as surfer movement may potentially bias the information being collected. Our project proposes the creation of a data processing framework to generate accurate wave statistic information from data collected by the Smartfin. Specifically, we perform spectral and wave-train analysis on Smartfin IMU data to determine significant wave height, wave period, and wave direction. This project will drastically increase the spatial density of oceanographic measurements.

Introduction

Understanding our oceans is incredibly important as our oceans produce most of the world’s oxygen, store more carbon dioxide than our atmosphere, and regulate earth’s climate. However, our oceans are severely undersampled as we rely on scientific measurements from infrequently spaced buoys, piers, and sub-surface moorings. Nearshore environments, like the surf zone, are particularly difficult to collect data in because high energy wave dynamics often break expensive scientific equipment. In order to increase the spatial density of oceanographic measurements, researchers at Scripps Institute of Oceanography have begun developing the Smartfin, a surfboard fin with embedded sensors capable of measuring multiple ocean parameters in nearshore environments. There are multiple sensors contained within the Smartfin, including a temperature sensor, a GPS, and a 9-axis Inertial Measurement Unit (IMU). The IMU alone consists of an accelerometer, gyroscope, and magnetometer. This project has the potential to vastly increase the spatial density of oceanographic measurements by making ocean data collected by the Smartfin’s sensors available to the worldwide scientific community. While the Smartfin’s hardware has been thoroughly tested by scientists at Scripps, very little data processing has been done. Therefore, this project involves the creation of a processing framework for meaningful analysis of data collected by the Smartfin. 

In particular, this project aims to calculate the same types of wave statistics that carefully calibrated CDIP buoys are capable of calculating. This includes computing significant wave height, wave direction, and wave period and frequency. In order to calculate significant wave height, both spectral analysis and wave train analysis methods are tested, while only spectral analysis is used to calculate wave period. For determining wave direction a new model was created that uses the raw magnetometer values given by the Smartfin’s 3-D compass and scales this vector using the accelerometer values. Plotting this vector on the X-Y axis gives the azimuth (heading) of the Smartfin, and averaging these values gives the average heading/direction of waves for the whole trip. To test accuracy of this model, experimental results from the Smartfin is compared with CDIP buoy data from the same day to get a percentage error for wave direction. The goal is to make sure this error is below 10%, in order to affirm the validity of the model. As an extension, the model will be tested with select data from surf sessions labeled using machine learning techniques. By selecting data specifically where the surfboard is only floating on the water, the model can be vetted for accuracy in real world scenarios.

In order to create a robust model that can accomplish both of the previous subgoals, an accurate way of determining the Smartfin’s orientation throughout the duration of each surf session is necessary. This in itself is pretty straightforward; given the accelerometer, gyroscopic, and magnetometer readings, one should be able to determine the amount of rotation (in terms of yaw, pitch, and roll angles) that the Smartfin undergoes during the surf session. However, the amount of rotation is relative to the initial orientation of the device, and herein lies our biggest difficulty: IMU data is useless unless one knows how to interpret it and relate the IMU’s frame of reference to a fixed, external reference frame, which in this case is the world’s frame of reference. The first thing we will need to do is to create a rotation matrix that aligns the sensor data such that the force of gravity points in the direction of the negative z-axis. After that, we can align the magnetic north to lie along the x-axis. The accelerometer outputs a gravity vector when the device is at a stable resting position (ideally upon starting the device), and simple physics demonstrates that the gravity vector always points down. We will find the matrix required to mathematically rotate the sensor so that the gravity vector lies along the negative z-axis. Once we complete this, we can follow a similar method for the magnetometer. Since the magnetometer outputs a magnetic north vector, we can find the matrix required to mathematically rotate the sensor so that the magnetic north vector lies along the x-axis. Therefore, we should theoretically be able to calculate the orientation of the Smartfin at any given time point during the surf session by using these rotation matrices to rotate the device into its proper frame of reference.

Another aim of this project is to map surfer location while they move about the ocean. We decided that this would be an extra-credit subproject that we would work on only if we had time. Even though the Smartfin contains a GPS, GPS signals do not propagate well underwater. Therefore, to compute the path the Smartfin traveled on, it is necessary to use the IMU data and Smartfin orientation code to predict surfer movement. Since all of the IMU data are oriented based on the IMU frame of reference, we transform these vectors to earth-centered, earth-coordinate frame (ECEF), by using the rotation matrix described above. Now with the direction of the accelerometer calibrated, we double integrate this vector in order to get the displacement, and then we use this displacement and direction in order to predict the next position of the surfer from the current position.

Technical Material

In order to complete the aforementioned tasks, our project was split into four subprojects. The first subproject was concerned with calculating significant wave height and wave period, the second with computing wave direction, the third with visualizing and mapping surfer location, and the fourth with determining the orientation of the Smartfin.

Section A. Calculating significant wave height and wave period

Spectral Analysis

Once we obtain the Smartfin’s vertical acceleration signal, we can perform both spectral analysis and wave train analysis, which are our two methodologies for determining significant wave height and wave period. These are our steps for performing spectral analysis: (1) detrend the data, (2) take the FFT of acceleration, (3) shift the FFT of acceleration, (4) normalize the FFT, (5) multiple the FFT signal by 1/w^2 where w= 2pif (f is the peak frequency that we got from the FFT of acceleration, in this case it was approx 0.16). After performing spectral analysis, we get the following graph after using the above acceleration signal as the input signal. The graph now shows vertical displacement in the frequency domain.

*Figure 1. Buoy Calibrator Experiment: Smartfin vertical vcceleration (left), same signal as vertical displacement in frequency domain (right) *

Table 1. Determining Peak Frequency and Wave Height Using Spectral Analysis on IMU Data

	Peak Frequency (Hz)	Wave Height (m)
Actual	0.160	1.80
Calculated	0.159	1.74
Standard Error	0.002	0.033
% Standard Error	0.20%	3.30%

Double Integration Wave Train Analysis

** **Our Wave Train Analysis uses the technique of doubly integrating our vertical acceleration data to obtain vertical displacement data. We can then use this data to determine significant wave height. Significant wave height is a statistical measurement for determining wave height from irregular waves; it is calculated as the mean of the largest one third, or 33%, of all waves.

Figure 2. Using a peak picking algorithm to determine the largest 1/3 of wave heights.

Table 2. Determining Significant Wave Height Using Double Integration on IMU Data Significant Wave Height [m] Actual 1.80 Calculated 1.88 Standard Error 0.044 % Standard Error 4.4%

Analysis 2: Benchmarking Pool Displacement Controlled Experiments

Figure 3. Vertical Acceleration vs. time for each of the controlled pool sub-experiments.

Table 4. Standard error calculated from wave height in each controlled pool experiment using Wave Train Analysis.

	Actual Wave Height [m]	Calculated Wave Height [m]	Standard Error	Standard Error %
Experiment #1	1.8	1.9848564307387153	0.10269801707	10.3%
Experiment #2	1.5	1.619954942662752	0.07996996177	8.00%
Experiment #3	0.9	0.8514080931935583	0.05399100756	5.40%

Analysis 3: Real World Ocean Data

Table 5. Floating on Ocean Near CDIP Buoy Wave Height [m] Wave Period [s] Actual (CDIP) 0.565 14.006 Calculated (Smartfin) 0.485 11.149 Error 0.142 0.204 % Error 14.2% 20.4%

Table 6. Analyzing Data from Real Surf Session Wave Height [m] Wave Period [s] Actual (CDIP) 0.60 9.0 Calculated (Smartfin) 0.22 9.5 Error 0.62 0.047 % Error 62% 4.7%

Section B. Wave Direction

Besides wave height, period and frequency, wave direction is also an essential aspect of wave behaviour that we need to quantify and model. This new model uses the raw magnetometer values given by the Smartfin’s 3-D compass and scales this vector using the accelerometer values. The raw values must first undergo a calibration process where the null readings are dropped, converted to the appropriate units and changed to the surfer’s reference frame. After calibrating these values, the next step is to calculate the azimuth and altitude of the trip, and then plot the azimuth against time. The following functions were used to calculate azimuth and altitude:

def azimuth(x,y,z):

real_y = y * (-1) # This is to account for y 

return (180/math.pi * math.atan2(real_y,x)) % 360

def altitude(x,y,z):

h = math.hypot(y, x)

return 180/math.pi * math.atan2(z,h)

Figure 1: Azimuth-time plot and polar representation

Once the azimuth and time graph can be visualized, it is clear that these values need to be averaged to get an average total heading for the entire trip. Theoretically, since these values represent the direction the Smartfin is being pushed at any time interval, the average heading should provide the direction of waves for this trip. The experimental average can then be compared to same day CDIP data to get a percentage error.

Figure 3: CDIP graphs for wave direction as reference

Comparing to CDIP data, my calculated average heading (deg) is: 286.96. The expected heading (deg) is: 282.41 and the percentage error in my model is: 0.016, or 1.6%. It is possible to further validate this model by testing with labeled data from real surf session data. Using labelled Smartfin footage, we can verify the accuracy of this algorithm on "floating" portions of a Smartfin surf session. After applying the same wave direction calculations mentioned above, a heading of 195.5 degrees [SW] was achieved, which needs to be validated with CDIP data from November 7th to get a percentage error. Overall, we believe that the model is accurate to a large degree, showing only 1.61% error for the CDIP buoy float experiment. As for real world data, this error still needs to be verified. The goal of creating a model with error below 10% was achieved. 

Section C. Smartfin Orientation

Analysis 1: Orientation Visualization using Simulation versus Recording of Surf Session

To get a baseline idea of how accurate our rotation matrix could determine the orientation of the Smartfin despite presence of ocean noise, our team used a Garmin Virb to conduct a video recording of the test surf session. This allowed us to visually identify the orientation of the Smartfin during the entire surf session. In an ideal world, we would be able to obtain the actual quaternion values of the Smartfin, and we would be able to do a numerical analysis on our rotation matrix and calculate the error rate of our methodology. However, we do not have an "answer key" to compare our numbers to, so this is the next best way of checking for accuracy. In this case, the GoPro was attached to the surfboard, which provides us with a first-person account of the orientation of the Smartfin. If the Garmin Virb turned upside down, we would know that our simulation should also reflect a flipping of the Smartfin. Sadly, we discovered that our rotation matrix is not accurate, as the simulation is displaying much jittering and erratic maneuvers.

Section D. Surfer Location and IMU Dead Reckoning

Analysis 1: Dead Reckoning Visualization using Algorithm Mapping versus Garmin Virb Mapping

       Figure 1. GPS Based Labels                    Figure 2. IMU Dead Reckoning Labels

Our goal is to create a model that would allow us to determine surfer position without using GPS coordinates. We want to move away from using an inconsistent GPS model (as shown in the image on the left) and create a model that will allow us to determine surfer position every 0.2 seconds. In order to determine how accurate our model is, our team used a Garmin Virb waterproof sports camera, which gave us the GPS location of the Smartfin in periodic intervals to create a mapping as shown on the left. We then took our dead reckoning map and created the second mapping as shown on the right. We then visually compared the output with the Garmin Virb output to check for accuracy (we can see the results of the two maps above).

Milestones

Section A. Updated Project Schedule*

	Overview	Deliverables	Who is Delivering?
Week 3 Thurs.	Finish Background Research	Project Specification Due	Jasmine
Week 4 Tues.	Refactoring Code	Smartfin Team Meeting Update	Jasmine, Brendon, Howard, Samprith
Week 4 Thurs.	Refactoring Code	Oral Project Update Presentation	Howard and Brendon
Week 5 Tues.	Fixing orientation, filtering, wave height and frequency code.	Updated code base	Brendon, Jasmine
Week 5 Thurs.	Start coding methodology for wave direction and IMU Dead Reckoning.	New Jupyter notebooks	Samprith, Howard
Week 6 Tues.	Milestone #1: Start Benchmarking tests in Controlled Environments	RD of Milestone #1 Report
Week 6 Thurs.	Milestone #1: Finish Benchmarking tests in Controlled Environments	Milestone Report: Written Test Results	Jasmine, Brendon, Howard, Samprith
Week 7 Tues.	Start Milestone #2: Testing in Real Ocean Environment (Data collection)	Data Collection from CDIP Buoy Float Experiment	Jasmine
Week 7 Thurs.	Finish coding methodology for wave direction and IMU Dead Reckoning.		Howard, Samprith
Week 8 Tues.	Fix bugs in Orientation code, test on lab Smartfin.		Brendon
Week 8 Thurs.	Finish Milestone #2: Testing in Real Ocean Environment (continued)	Written Report of Test Results	Jasmine, Brendon, Howard, Samprith
Week 9 Tues.	More coding: fix bugs/errors, make code more robust for noisier dataset.	Update reports, new solutions to old problems in code	Jasmine, Brendon, Howard, Samprith
Week 9 Thurs.	Start writing final report. Start filming video project.	Storyboard rough draft and final report outline.	Jasmine, Brendon, Howard, Samprith
Week 10 Tues.	Continue filming video project.
Week 10 Thurs.	Finish video project and final report.	Final Report, Final Video Due	Jasmine, Brendon, Howard, Samprith

• Our project schedule was updated slightly from our original project specification to include more detailed goal overviews and deliverables; for example, Milestone #1 was changed to include benchmarking previous results and code refactoring. Additionally weeks 9-10 were updated to include class project tasks, such as filming the final video project. Also tasks were changed around to be more representative of what was accomplished each week.

Section B. Overview of Milestones

Part 1. Significant Wave Height and Wave Period Milestones

For the significant wave height and wave period subproject, the main milestones were:

1. Benchmarking results from an earlier controlled experiments where waves were simulated by a CDIP buoy calibrator.

2. Benchmarking results from an earlier controlled experiments where waves were simulated manually in a pool by displacing the Smartfin at specific vertical intervals.

3. Computing significant wave height and wave period from a real-world experiment where we floated surfboards holding Smartfins on the ocean near a CDIP buoy.

4. And finally, computing significant wave height and wave period from real surf sessions, where the surf sessions were videotaped and surfer motion at 1s time intervals was recorded.

Part 2. Wave Direction Milestones

For wave direction, the main milestones were:

1. Develop a new model of calculating wave direction using Smartfin IMU.

2. Get <10% error using Smartfin buoy float experiment data compared to CDIP buoy data.

3. Verify model on real surf session data by labeling surf session and selecting test values.

Part 3. Smartfin Orientation Milestones

For the orientation subproject, the main milestones were:

1. Create a rotation matrix using the initial readings from the accelerometer and magnetometer.

2. Calculate quaternion values for the Smartfin orientation throughout the surf session.

3. Simulate the surf session using the quaternion values.

4. Use a video recording of the surf session to visually compare the accuracy of the simulation.

Part 4. Surfer Location and IMU Dead Reckoning Milestones

For the dead reckoning extra-credit subproject, the main milestones were:

1. Create a GPS mapping of surf session data.

2. Integrate rotation matrix code in order to transform IMU data from IMU frame of reference to ECEF frame of reference.

3. Use IMU data (oriented to ECEF) in order to predict surfer position at a rate of 5 Hz.

Milestone Completion:

Almost all of these milestones were accomplished and their results are included in the previous "technical material" section of this paper; the only milestones that were not accomplished include: (1) the rotation matrix that we are using does not appear to be very accurate, and (2) we still need to verify the wave direction model on CDIP data taken from a same-day surf session.

Conclusion

In conclusion, we were able to accomplish all of the main goals that we set out to accomplish for this project. For the Wave Height and Wave Period subgoal, we were able to benchmark our previous experiments which included computing significant wave height and peak wave period. This process allowed us to quantify how we expect our algorithms to perform in the best case scenario. From there, we experimented on real world ocean data in order to see how our algorithms performed in actual ocean conditions, which were much more variable and much more realistic than our experiments. Although this introduced more error into our calculations, these experiments helped validate our algorithms in real world ocean conditions. The final experiment that was performed for this subgoal was computing wave height and wave period from a real surf session. We decided to calculate these statistics specifically from portions of a surfer’s surf session where the surfer is floating. Even though we got very accurate results for wave period (4.7% error), we received very inaccurate results for significant wave height (62% error). In the future, we will need to run more experiments on surf sessions as well as test our accuracy calculating each statistic during different types of surfer motion, such as paddling and surfing. Overall, successful completion of this subgoal will allow oceanographers to crowdsource the computation of wave height and wave period in nearshore environments by utilizing surfers. 

Our Wave Direction model behaved as expected, and we were able to successfully calculate an average heading using azimuth values. With an error of less than 10% this exceeded our expectations by a great margin. After testing this model further in labeled surf data however, it became evident that the noise levels present in this new data was worse than expected. There are likely more accurate ways to calculate wave direction from real surf data that we need to research and use to improve our model in the future. Overall, we achieved our goal and are optimistic about the future for our model.

For the Smartfin Orientation subgoal, we were able to create some sort of rotation matrix that allowed us to somewhat visualize the orientation of the Smartfin throughout a given surf session. Although this iteration of orientation code is inaccurate, it is a starting point for future work on orientation, and it at least tells us what not to do, and which reference works we can use as inspiration.



Furthermore for the SmartFin Dead-Reckoning subgoal, we were able to create a model that estimates the surfer position at a rate of 5 Hz, that we were able to verify comparing it to the GPS plot from the Garmin Virb camera that we mounted on top of the surfboard to record the session. Though we must be aware of the error that propagates through the various layers of the model (Rotation transformations, double integration etc), and seek to make the model more accurate in future work.

References

L. C. BENDER III, N. L. GUINASSO JR., AND J. N. WALPERT. "A Comparison of Methods for Determining Significant Wave Heights." Journal of Atmospheric and Oceanic Technologies. June 2010.https://journals.ametsoc.org/doi/pdf/10.1175/2010JTECHO724.1

Hughes, Mark. "How to Interpret IMU Sensor Data for Dead-Reckoning: Rotation Matrix Creation." All About

Circuits, 23 Apr. 2019.

https://www.allaboutcircuits.com/technical-articles/how-to-interpret-IMU-sensor-data-dead-reckoning-rotation-matrix-creation/

Datawell BV Oceanographic instruments. "Datawell Waverider Reference Manual". October 10 2009.

http://m.cdip.ucsd.edu/documents/index/gauge_docs/mk3.df

Manon Kok, Jeroen D. Hol and Thomas B. Sch ̈on (2017), "Using Inertial Sensors for Position and Orientation

Estimation", Foundations and Trends in Signal Processing: Vol. 11: No. 1-2, pp 1-153.

http://dx.doi.org/10.1561/2000000094

Marshall D. Earle. "Nondirectional and directional wave data analysis procedures". Stennis Space Center, January 1996. https://www.ndbc.noaa.gov/wavemeas.pdf

CDIP. "073 - SCRIPPS PIER, LA JOLLA CA." *June 2019. *https://cdip.ucsd.edu/?nav=recent&sub=observed&stn=073&stream=p1&xitem=splot