FixErrors Usage

FixErrors Matlab GUI

The FixErrors Matlab GUI identifies frames in which the tracker may have performed poorly, shows these to you, and allows you to correct any mistakes. Suspicious frames and flies are any for which

• A fly's trajectory ends.
• A fly's trajectory begins.
• The cost of swapping the identities of a pair of flies is small (i.e., the right identity assignments are not obvious).
• The constant-velocity motion model poorly predicts the trajectories (e.g., a fly jumps).
• The major axis length for a fly is large.
• The orientation of a fly changes a lot from one frame to the next.
• The direction of a fly's velocity and its orientation do not match (i.e., a fly appears to be walking backward).

These frames are shown in order of most to least suspicious. The user then has the option to modify the fit trajectories to correct any mistakes in the tracking. Empirically, we found that all identity errors were easily located and fixed using this GUI.

---

Contents

• Start-Up
• Suspiciousness Parameters
• FixErrors GUI Display
• Manipulating Trajectories
  • Edit Tools: Delete Track
  • Edit Tools: Swap Identities
  • Edit Tools: Connect Tracks
  • Edit Tools: Interpolate
  • Edit Tools: Extend Track
  • Edit Tools: Auto Track
  • Edit Tools: Flip Orientation
  • Edit Tools: Auto Track Multiple

---

FixErrors Start-Up

FixErrors begins by prompting you for the name of the raw movie whose trajectories are to be checked, followed by the name of the MAT-file containing the trajectories to be checked and the name of the corresponding Ctrax annotation file. If the trajectories have not yet been augmented with the pixel-to-millimeter and frame-to-second conversions, the convert_units.m script (if available) is run to get these. FixErrors then checks to see if it has already been run on this movie and there is saved information available to pick up where it left off. If it finds such saved information, it will prompt to see if you want to continue.

---

FixErrors Suspiciousness Parameters

If FixErrors has never been run on this movie or you choose not to continue, you are then prompted for parameters used in detecting "suspicious" trajectory sequences -- these are sequences in which tracking is more likely to have failed. The suspiciousness parameters are the following:

• Minimum suspicious prediction - detection error (mm): This parameter is used to identify sequences in which the constant-velocity motion model poorly predicts the trajectories. All sequences in which the error between the constant-velocity prediction and the measured positions is greater than the set value will be flagged. By default, this is initialized to one-fifth the "Max Jump Error" from the Ctrax annotation file.
• Minimum suspicious orientation change (deg): All sequences in which the change in orientation is greater than the set value will be flagged. We use 45 degrees in our experiments.
• Minimum suspiciously large major axis (mm): All sequences in which the major axis length is greater than the set value will be flagged. For reference, the max and mean major axis lengths read from the Ctrax annotation file are shown.
• Minimum suspicious orientation - velocity direction mismatch (deg) and Minimum walking speed (mm/frame): All sequences in which the fly is walking with speed greater than this minimum walking speed and the orientation and velocity direction mismatch is more than above value will be flagged.
• Maximum ambiguous error (mm^2): All sequences in which the increase in error for swapping a pair of identities is less than the set value will be flagged.

Note that it is okay to set these suspiciousness parameters to be overly cautious, i.e., so that a lot of sequences are flagged. The FixErrors GUI will show the sequences in order from most suspicious to least suspicious sequence of a given type. Once you feel that the rest of the sequences of a given type are probably correct, you can move on to another type of suspicious sequence or quit the program altogether.

Once these parameters are set, FixErrors goes through the trajectories and identifies the suspicious sequences. Depending on the number and length of the trajectories, this may take some time. When it is done, it initializes the MAT-file containing the current/ongoing results of FixErrors (used to restart the GUI), then brings up the FixErrors GUI itself.

---

GUI Display

The left side of the GUI shows the current frame annotated with the trajectories of the fly. For each fly, the fit ellipse is shown. Tail direction is indicated by a line segment. Using the "Plot Path" drop-down menu, one can select whether the paths of "All Flies", "Seq Flies" (only those flies that are part of the current suspicious sequence), or "No Flies" are plotted. The number of frames of trajectory around the current frame that are plotted can be adjusted with the "NFrames Plot" control. The "Zoom" drop-down menu controls whether FixErrors zooms in on the flies in the current sequence or shows the whole arena.

The "Sequence Info" box shows information about the current suspicious sequence. The "Error" number shows which error sequence of this type is shown (this is not indexed by suspiciousness, but rather fly and frame). It shows the numbers of the frames and the identities of the flies that are suspicious ("Frames" and "Flies"). It shows the "Type" of suspicious sequence, and finally the suspiciousness of the sequence ("Susp"). The "suspiciousness" measure varies from one type of error to another, but will always be nonnegative for the selected sequences, and zero for the least suspicious sequence selectable.

The "Frame Info" box shows information about the current frame and flies. It shows the (editable) current frame number, which frame of the current sequence this is, the suspiciousness for the current frame (the suspiciousness of the sequence is the maximum per-frame suspiciousness over all frames in the sequence). The "Selected Fly" shows the identity of the last clicked fly. This is useful if the colors of flies being examined are similar.

The "Navigation Tools" control the order in which suspicious sequences are shown. Here, you can select the next type of error to show and whether the sequences should be sorted by suspiciousness, fly, or frame. We recommend sorting by suspiciousness. When you are satisfied that the trajectories are correct, hit the "Correct" button to go on to the next sequence. The "Back" button is not yet implemented, the "Save" button saves the current state of the GUI to the restart file, and the "Quit" button exits the GUI.

The "Seek Tools" allow you to change the current frame to the next or previous track birth or death in the currently selected axes.

Manipulating Trajectories

The positions of the flies in the current frame can be manipulated by dragging around the plotted ellipses. The center position of the ellipse can be changed by dragging the white circle at the center of the ellipse. The white circles at the four end points of the fly's axes can be dragged independently to set these points.

For more drastic changes to the trajectories, use the following "Edit Tools":

Edit Tools: Delete Track

Deletes the trajectory of a selected fly from the selected frame to the end of its track. To delete the fly's entire trajectory, you would want to delete from the first frame of its trajectory. Follow these steps to delete a trajectory portion:

1. Scroll to the first frame of the portion of track to be deleted.
2. Click on the fly to be deleted.
3. Push the "Do It" button.

Edit Tools: Swap Identities

Swaps the identities of a selected pair of flies from a selected frame to the end of their tracks. Follow these steps:

1. Scroll to the first frame of the portion of tracks to be swapped.
2. Select the two flies to be swapped.
3. Push the "Do It" button.

Edit Tools: Connect Tracks

Connects the end of one trajectory to the start of another (useful if, e.g., a fly is lost for some number of frames and then reborn as a new fly identity). Follow these steps:

1. Scroll to the first frame of the portion of track to be connected.
2. Select the first fly to be connected.
3. Push the "First Fly" button.
4. Scroll to the last frame of the portion of track to be connected.
5. Click on the second fly to be connected to.
6. Push the "Do It" button.

If neither of the fly tracks is "alive" during a subset of the frames, the position of the fly is set by linearly interpolating between its start and end locations. If both flies are alive during a portion of the frames, the position of the fly is set as the average of these two.

Edit Tools: Interpolate

Replaces the positions and orientations of a single fly over a sequence of frames using linear interpolation between the start frame and the end frame. Follow these steps.

1. Scroll to the first frame of the portion of track to be interpolated.
2. Click on the fly to be interpolated.
3. Push the "First Frame" button to select.
4. Scroll to the last frame of the portion of track to be interpolated.
5. Push the "Do It" button.

Edit Tools: Extend Track

Extends a selected fly's track in time from the end of its track to a selected frame. Follow these steps.

1. Click on the fly to be extended.
2. Push the "First Fly" button to select it.
3. Scroll to the last frame to extend until.
4. Push the "Do It" button.

From the end of the fly's trajectory to the selected frame, the fly will be set to be in the position in its original last frame (useful if, e.g., a fly's track dies because it stops moving and becomes part of the background).

Edit Tools: Auto Track

Modifies a sequence of frames for a single fly in one step. During auto-tracking, FixErrors will remove all foreground pixels that can be attributed to other flies. Then, it finds the closest connected component of remaining foreground pixels and stores the ellipse fit to this component as the position of the fly. There are a few parameters that can be modified in the auto-tracking. First, you can fix errors in the background modeling by selecting regions of the frame and temporarily filling them with a selected color. Second, you can set the "Track Radius" -- the size of the square around the fly's predicted position that is examined during the auto-tracking. Third, you can temporarily alter the background-subtraction threshold. To automatically retrack a fly for a sequence of frames:

1. Click on the fly to be tracked in the frame to be tracked from.
2. Push the "First Frame" button to select this fly and frame.
3. If desired, modify the tracking settings by pushing the "Settings..." button:
1. The image panel shows the first frame in a window around the selected fly's position in that frame. In red, it outlines the connected components of foreground pixels detected with the current settings.
2. To fix the background model, drag a rectangle in the "retrack_settings" dialog image. Then, click the "Eyedropper" button, and click on the image to select the color to fill the rectangle with. Finally, click the "Fill" button.
3. The window shown reflects the "Track Radius". The "Track Radius" can be modified with the corresponding "+" and "-" buttons.
4. The background subtraction "Threshold" can be modified with the corresponding "+" and "-" buttons.
5. Specify that the flies are light on a dark background, dark on a light background, or other.
6. Click the "Done" button when finished modifying the tracking settings.
4. To see the tracking results as they are computed, select "Show Tracking". For faster tracking, deselect it.
5. Scroll to the frame to be tracked until, then click the "Do It" button.
6. To stop tracking at any time, hit the "Stop" button.

Edit Tools: Flip Orientation

Flip the head/tail assignment for a selected fly for an interval (i.e. the fly is rotated by 180 degrees) by following these steps:

1. Click on the fly to be flipped in the first frame to be flipped.
2. Push the "First Frame" button to select.
3. Scroll to the last frame to be flipped until.
4. Push the "Do It" button.

Edit Tools: Auto Track Multiple

Similar to Edit Tools: Auto Track, you can re-track multiple flies for which the splitting of a single connected component into multiple flies has failed. As in auto-tracking, FixErrors removes the foreground pixels that can be attributed to flies other than those being retracked. It then fits a mixture of Gaussians to the remaining foreground pixels, initializing with the predicted positions of the flies. This can sometimes perform better than the Ctrax tracker, as it forces a given number of flies to be fit and can use the previous positions of the flies for initialization. To automatically retrack a set of flies for a sequence of frames:

1. Click on the flies to be tracked in the first frame to be tracked.
2. Push the "First Frame" button to select these flies and frame.
3. If desired, modify the tracking settings by pushing the "Settings..." button:
   1. The image panel shows the first frame in a window around the selected fly's position in that frame. In red, it outlines the connected components of foreground pixels detected with the current settings.
   2. To fix the background model, drag a rectangle in the "retrack_settings" dialog image. Then, click the "Eyedropper" button, and click on the image to select the color to fill the rectangle with. Finally, click the "Fill" button.
   3. The window shown reflects the "Track Radius". The "Track Radius" can be modified with the corresponding "+" and "-" buttons.
   4. The background subtraction "Threshold" can be modified with the corresponding "+" and "-" buttons.
   5. Specify that the flies are light on a dark background, dark on a light background, or other.
   6. Click the "Done" button when finished modifying the tracking settings.
4. To see the tracking results as they are computed, select "Show Tracking". For faster tracking, deselect it.
5. Scroll to the frame to be tracked until, then click the "Do It" button.
6. To stop tracking at any time, hit the "Stop" button.