DLC Centroid populate error with spatial_series

[rpswenson]

Got a new error when going through the DLC pipeline (which means that the changes in today's PR worked, yay). When running sgp.DLCCentroid.populate(centroid_key) I run into the error:

Error stack

Cell In[12], line 71
     69 centroid_key["dlc_centroid_params_name"] = centroid_params_name
     70 sgp.DLCCentroidSelection.insert1(centroid_key, skip_duplicates=True)
---> 71 sgp.DLCCentroid.populate(centroid_key)
     72 (sgp.DLCCentroid() & centroid_key).fetch1_dataframe().plot.scatter(
     73     x="position_x",
     74     y="position_y",
   (...)
     79     figsize=(10, 10),
     80 )
     82 ##DLC Orientation

File ~/datajoint-python/datajoint/autopopulate.py:247, in AutoPopulate.populate(self, suppress_errors, return_exception_objects, reserve_jobs, order, limit, max_calls, display_progress, processes, make_kwargs, *restrictions)
    241 if processes == 1:
    242     for key in (
    243         tqdm(keys, desc=self.__class__.__name__)
    244         if display_progress
    245         else keys
    246     ):
--> 247         status = self._populate1(key, jobs, **populate_kwargs)
    248         if status is True:
    249             success_list.append(1)

File ~/datajoint-python/datajoint/autopopulate.py:314, in AutoPopulate._populate1(self, key, jobs, suppress_errors, return_exception_objects, make_kwargs)
    312 self.__class__._allow_insert = True
    313 try:
--> 314     make(dict(key), **(make_kwargs or {}))
    315 except (KeyboardInterrupt, SystemExit, Exception) as error:
    316     try:

File ~/spyglass/src/spyglass/position/v1/position_dlc_centroid.py:301, in DLCCentroid.make(self, key)
    283 METERS_PER_CM = 0.01
    284 position.create_spatial_series(
    285     name="position",
    286     timestamps=final_df.index.to_numpy(),
   (...)
    291     description="x_position, y_position",
    292 )
    293 velocity.create_timeseries(
    294     name="velocity",
    295     timestamps=velocity_df.index.to_numpy(),
    296     conversion=METERS_PER_CM,
    297     unit="m/s",
    298     data=velocity_df.loc[
    299         :, idx[("velocity_x", "velocity_y", "speed")]
    300     ].to_numpy(),
--> 301     comments=spatial_series.comments,
    302     description="x_velocity, y_velocity, speed",
    303 )
    304 velocity.create_timeseries(
    305     name="video_frame_ind",
    306     unit="index",
   (...)
    312     comments="no comments",
    313 )
    314 nwb_analysis_file = AnalysisNwbfile()

UnboundLocalError: local variable 'spatial_series' referenced before assignment

The centroid key is:

{'nwb_file_name': 'BS2820231107_.nwb',
 'epoch': 2,
 'video_file_num': 1,
 'project_name': 'BS2820231107',
 'dlc_model_name': 'BS2820231107_tutorial_00',
 'dlc_model_params_name': 'default',
 'dlc_si_cohort_selection_name': 'green_red_led',
 'dlc_centroid_params_name': 'default'}

[samuelbray32]

Thanks Rio. Missed that one. If you don't mind pulling from the dlc_no_pos branch and continuing, just post any other errors you hit on this issue thread to keep it organized. Thanks!

[rpswenson]

Ok, I'm not sure if this is actually related since it has gone through one full loop already, but since it is also using the spatial_series variable it might be. Running sgp.DLCPoseEstimation().populate(pose_estimation_key) with pose_estimation_key being

{'nwb_file_name': 'BS2820231107_.nwb',
 'epoch': 4,
 'video_file_num': 1,
 'project_name': 'BS2820231107',
 'dlc_model_name': 'BS2820231107_tutorial_00',
 'dlc_model_params_name': 'default',
 'task_mode': 'trigger'}

gets this error:

Cell In[12], line 22
      8 pose_estimation_key = (
      9     sgp.DLCPoseEstimationSelection.insert_estimation_task(
     10         {
   (...)
     18     )
     19 )
     21 ##populate DLC Pose Estimation
---> 22 sgp.DLCPoseEstimation().populate(pose_estimation_key)
     24 ##start smooth interpolation
     25 si_params_name = "just_nan"

File ~/datajoint-python/datajoint/autopopulate.py:247, in AutoPopulate.populate(self, suppress_errors, return_exception_objects, reserve_jobs, order, limit, max_calls, display_progress, processes, make_kwargs, *restrictions)
    241 if processes == 1:
    242     for key in (
    243         tqdm(keys, desc=self.__class__.__name__)
    244         if display_progress
    245         else keys
    246     ):
--> 247         status = self._populate1(key, jobs, **populate_kwargs)
    248         if status is True:
    249             success_list.append(1)

File ~/datajoint-python/datajoint/autopopulate.py:314, in AutoPopulate._populate1(self, key, jobs, suppress_errors, return_exception_objects, make_kwargs)
    312 self.__class__._allow_insert = True
    313 try:
--> 314     make(dict(key), **(make_kwargs or {}))
    315 except (KeyboardInterrupt, SystemExit, Exception) as error:
    316     try:

File ~/spyglass/src/spyglass/position/v1/position_dlc_pose_estimation.py:299, in DLCPoseEstimation.make(self, key)
    292 position = pynwb.behavior.Position()
    293 likelihood = pynwb.behavior.BehavioralTimeSeries()
    294 position.create_spatial_series(
    295     name="position",
    296     timestamps=part_df.time.to_numpy(),
    297     conversion=METERS_PER_CM,
    298     data=part_df.loc[:, idx[("x", "y")]].to_numpy(),
--> 299     reference_frame=get_video_path(
    300         spatial_series, "reference_frame", ""
    301     ),
    302     comments=getattr(spatial_series, "comments", "no commwnts"),
    303     description="x_position, y_position",
    304 )
    305 likelihood.create_timeseries(
    306     name="likelihood",
    307     timestamps=part_df.time.to_numpy(),
   (...)
    311     description="likelihood",
    312 )
    313 likelihood.create_timeseries(
    314     name="video_frame_ind",
    315     timestamps=part_df.time.to_numpy(),
   (...)
    319     description="video_frame_ind",
    320 )

TypeError: get_video_path() takes 1 positional argument but 3 were given

[samuelbray32]

Yeah, was just me multitasking and not paying enough attention to the tab complete. I can't run becasue of permissions, but try the new commit on dlc_no_pos again please. Thanks!

[rpswenson]

Got through the full loop without any new errors! Thanks!

[rpswenson]

Trying to get a video from the dlc output and ran into something that Eric and Mike said I should bring here. Running sgp.DLCPosVideo().populate(dlc_key) trips the error: AttributeError: 'list' object has no attribute 'replace' because at the time of the error, the key has been modified:

File ~/spyglass/src/spyglass/position/v1/position_dlc_selection.py:326, in DLCPosVideo.make(self, key)
    314 interval_list_name = (
    315     convert_epoch_interval_name_to_position_interval_name(
    316         {
   (...)
    321     )
    322 )
    323 key["interval_list_name"] = interval_list_name
    324 epoch = (
    325     int(
--> 326         key["interval_list_name"]
    327         .replace("pos ", "")
    328         .replace(" valid times", "")
    329     )

and the interval_list_name that it is trying to pull is an empty list rather than a string:

ipdb>  key

{'dlc_si_cohort_centroid': 'green_red_led', 'nwb_file_name': 'BS2820231107_.nwb', 'epoch': 12, 'video_file_num': 1, 'project_name': 'BS2820231107', 'dlc_model_name': 'BS2820231107_tutorial_00', 'dlc_model_params_name': 'default', 'dlc_centroid_params_name': 'default', 'dlc_si_cohort_orientation': 'green_red_led', 'dlc_orientation_params_name': 'default', 'dlc_pos_video_params_name': 'default', 'interval_list_name': []}

We think that due to leaving out trodes position, the nwb has not generated any pos_intervals. Would we be able to use the dlc output to generate those missing pos_intervals?