Basal calories on Android

[bryantee]

I'm trying to get calories.active and they're coming back null on GoogleFit. While debugging I discovered that when the subsequent calories.basal queryAggregated gets called, the data returned also has a value of null.

My data coming back looks like this from calories.basal:

query data: [
	{
		"startDate": "2017-12-24T07:00:00.000Z",
		"endDate": "2017-12-25T06:53:36.140Z",
		"sourceBundleId": "com.google.android.gms",
		"value": null,
		"unit": "kcal"
	},
	{
		"startDate": "2017-12-25T06:53:36.140Z",
		"endDate": "2017-12-26T06:53:36.140Z",
		"sourceBundleId": "com.google.android.gms",
		"value": null,
		"unit": "kcal"
	},
	{
		"startDate": "2017-12-26T06:53:36.140Z",
		"endDate": "2017-12-27T06:53:36.140Z",
		"sourceBundleId": "com.google.android.gms",
		"value": null,
		"unit": "kcal"
	}, ...
]

My query is just wrapped in a promise but looks like:

return new Promise((resolve, reject) => {
            (<any>navigator).health.query({
                startDate: start,
                endDate: end,
                dataType: healthDataType,
                ...bucket && { bucket }
            }, resolve, reject);
});

I've tried various bucket types, 'day', 'week', 'month'

Data and steps seems to be coming back fine. Anyone have any ideas? Am I forming my query incorrectly? Have I found a bug?

[ghost]

GF has no separate query for basal and active calories, so I need to get basal separately and remove them from the total. That sounds easy, but I have experienced issues with basal calories because of the way GF treats them.
Basal calories are not always present for all days, so I query for basal calories over a week and average the results per day and use that as a reference. The week period I've found to be a reasonable compromise that has always worked with me, but it may be failing in your case.

Try to query the basal calories over the week under your analysis and see if there's anything there.

In any case it's strange that you get null, because the variable is initialised as 0, so there may be a bug introduced by some more recent commit? Let me know!

[bryantee]

Thanks @dariosalviwork. I'll try again today, though I'm fairly sure bucket of 'week' yielded the same results. I'll get back with my findings shortly.

UPDATE 2:

Ok, so if you look at the data below, calories.basal query is coming back 0. So obviously var basal_ms = data.value / (opts.endDate - opts.startDate); is going to come back NaN. So the question now is, why are we not getting any calories.basal?

UPDATE:

I just tried using bucket 'week' and still seem to be getting null.

09:41:54 start querying feature: Calories
09:41:55 calories.basal: 0
09:41:55 bucket: week
09:41:55 basal_ms: NaN
09:41:55 Calories: results!
09:41:55 {
	"startDate": "2018-01-13T19:00:00.000Z",
	"endDate": "2018-01-13T21:30:05.000Z",
	"sourceBundleId": "com.google.android.apps.fitness",
	"value": null,
	"unit": "kcal"
}
09:41:55 {
	"startDate": "2018-01-14T19:00:00.000Z",
	"endDate": "2018-01-14T20:05:00.000Z",
	"sourceBundleId": "com.google.android.apps.fitness",
	"value": null,
	"unit": "kcal"
}
09:41:55 {
	"startDate": "2018-01-17T16:00:00.000Z",
	"endDate": "2018-01-17T16:20:00.000Z",
	"sourceBundleId": "com.google.android.apps.fitness",
	"value": null,
	"unit": "kcal"
}...

Strangely enough, I'm comparing to Google Fit data, and date and times returned actually correspond to activities where calories are recorded. But the actual value isn't there. I'll keep investigating.

[dariosalvi78]

please try with non aggregated queries to see what's there.

The basal is computed only on one week (7 days - the end timestamp). Maybe there is data before that but it's not picking it up.
If that's the case, the solution would be just extending that window of time, but it needs a little change in the Java code.

[bryantee]

Thanks for the help so far @dariosalvi78

Doing just a non-aggregated query for calories.basal is returning me an empty array.

Meanwhile, querying the last 7 days of calories works fine:

16:25:36 calories:  [
	{
		"startDate": "2018-01-17T16:00:00.000Z",
		"endDate": "2018-01-17T16:20:00.000Z",
		"sourceBundleId": "com.google.android.apps.fitness",
		"value": 200,
		"unit": "kcal"
	},
	{
		"startDate": "2018-01-17T16:20:00.000Z",
		"endDate": "2018-01-17T17:00:00.000Z",
		"sourceBundleId": "com.google.android.gms",
		"value": 48.666664123535156,
		"unit": "kcal"
	},
	{
		"startDate": "2018-01-17T17:00:00.000Z",
		"endDate": "2018-01-17T17:30:00.000Z",
		"sourceBundleId": "com.google.android.gms",
		"value": 36.499996185302734,
		"unit": "kcal"
	},...

queryAggregated() is returning values of 0. Maybe there is no basal calories stored?:

calories.basal:  [
	{
		"startDate": "2018-01-17T07:00:00.000Z",
		"endDate": "2018-01-18T07:00:00.000Z",
		"value": 0
	},
	{
		"startDate": "2018-01-18T07:00:00.000Z",
		"endDate": "2018-01-19T07:00:00.000Z",
		"value": 0
	},
	{
		"startDate": "2018-01-19T07:00:00.000Z",
		"endDate": "2018-01-20T07:00:00.000Z",
		"value": 0
	},

[ghost]

yes, so that's the issue. It can't figure out what the basal are.
Can you, please, check when are the last basal present? I'll modify the java code accordingly.

BTW, it may be the case that basal are never available until you fill-in weight and/or height. Check that too please.

This basal calories story has been bothering me for quite some time...

[bryantee]

Gotcha. Height and weight are filled in.

I'm currently working around by querying activity by day and looping through to come up with total active calories per day. Might not be perfect, but seems to be doing the trick.

[ghost]

so the problem was that there were no weight and height?

If that's the case I'll update the docs to warn people....

[bryantee]

No, to clarify, height and weight were filled in and we're still not getting any data back for basal.

The work around was to query on activities and pull calories off that to arrive at some kind of "Active calories."

[dariosalvi78]

alternatively, you can try computing basal calories yourself with a formula like this:
http://dailyburn.com/life/health/how-to-calculate-bmr/

Actually it could be implemented in Java, as a fall back in case it can't find the basal.

[cooliscool]

I was trying to debug the basal issue. Finally reached here.

DataReadRequest req = new DataReadRequest.Builder()
                .aggregate(DataType.TYPE_BASAL_METABOLIC_RATE, DataType.AGGREGATE_BASAL_METABOLIC_RATE_SUMMARY)
                .bucketByTime(1, TimeUnit.HOURS)
                .setTimeRange(startTime, endTime, TimeUnit.MILLISECONDS)
                .build();

Had to start the time range from 4 weeks back to get a single aggregated data point.
In my case, Height&Weight was filled long back, also Calories expended in google fit shows sum of both Active and basal.

Can't find a reliable way to separate active and basal calories from total. Let me know if you have any solution to this.

[dariosalvi78]

this is a known issue, basal calories are present only when Google Fit decides to compute them, and we don't know when this happens. The function getBasalAVG() scans an entire year to find a samples and averages them.

[cooliscool]

After spending couple of days on this, I have reached the following conclusion :

If someone has recorded their height,weight,age in Google Fit, then a basal MR entry is created at that instant. Any change height/weight/age will trigger a new basal MR entry in Fit Store.

So, the latest value of BMR can be obtained through a simple query(), depending on when user have recorded the above parameters.

I found out this to be the best way to separate active calories from basal. Finding all samples and doing their average would cause negative values, as you're saying here

I'd like to take a moment to appreciate your effort on this project @dariosalvi78 . Thanks.

[dariosalvi78]

hey, it's very good that you've found that out! Unfortunately we don't know when exactly people input that data, so the problem is still there. Based on what you say, though, maybe it makes more sense to get the latetst basal value rather than the average?
Happy to receive PR with improvements!

[juanmaldonadodev]

@cooliscool @dariosalviwork @bryantee Can you check this issue?
#295
My proposal would be to change calculation of calories.active by
1 Allow get calories from activities without request distance or
2 Change logic of obtain calories.active and instead of query basal. Query activities with calories.

[dariosalvi78]

@juanico18 calories retrieved with activities are total, not active, see code from here. Your proposal wouldn't solve it unfortunately.

[juanmaldonadodev]

@dariosalvi78 Ok, I did some test and check in other apps that get that value as calories active. Although the type is calories expended as they are associated to an activity I did not think that Google take into account in the calculation of calories for an activity the basal value.
Have you checked if the result is the same by substract the basal than for sum all the calories of the activities?

[dariosalvi78]

the query that is issued to retrieve calories associated to activities is, without any doubt, related to the total calories, see the source code as pointed above. See the definition of TYPE_CALORIES_EXPENDED that Google gives here. I quote:

Note: this total calories number includes BMR calories expended.

Now, if Google Fit sends back only the active ones, it's a bug within Google Fit. If you are pretty sure that Google Fit always ignores basal calories, then you can just query for total ones. One question you should ask yourself: do you really need the active calories?

[juanmaldonadodev]

Yes we need them. Some calorie counter applications as LifeSum or MyFitnesspal when you active the integration with Google Fit they just display active calories. Because the basal calories are calculated with de user data in the own app and not with the information of Google Fit

[dariosalvi78]

Bear in mind that the calories that we retrieve per activity are based on the timestamps, not on the source of the input. This means that there is no explicit connection between the activity, as inserted by another app, and the calories. It is well possible that 2 apps write 2 activities on the same time range and the calories get summed up.

If you need to retrieve the calories as they are input by a specific app (or all apps except GF), then there is a way to filter by source in the Fitness APIs, but this is not integrated in this plugin.

With this plugin I can think of only 2 possibilities:
a) you do get total calories and ignore the problem altogether
b) you query all calories within a certain time frame, unaggregated, and filter according to the source ID (and match the corresponding activity based on source ID and time, if you need activities)

The second option may give you the exact amount as you get on the other apps, but it's costly (if the time span is large you can get a big array). To be clear: querying activities with calories and distance may be even more costly, because it issues a new query per each activity retrieved.