3 - Background tab

Simon Garnier

Source: vignettes/z3_background.Rmd

This page applies to trackR("classic") and trackR("quanta"). Skip it if you’re using trackR("tracktor").

trackR can use two different background subtraction methods in order to isolate objects from their surrounding background:

  1. “Classic” background subtraction: this method consists in subtracting an image representing a fixed background from each frame of the video. Pixels with high difference values in the resulting image will correspond to objects moving in front of the background.
  2. “Quantized” background subtraction: this method consists in identifying pixels that are significantly lighter or darker than the usual shade at their location in the image. This is accomplished by estimating for each pixel the lower and upper bounds of the confidence interval of its distribution of shades.
    If for a given image the pixel value falls outside of one of these bounds, then the pixel is considered as belonging to an object and not to the background.

For instructions on “classic” background subtraction, go to “Classic” background subtraction.

For instructions on “quantized” background subtraction, go to “Quantized” background subtraction

1 - “Classic” background subtraction

1 - Using an existing background image

If you have recorded a background image of your experimental setup, or if you have created and saved one following the instructions in the next section, you can load it by clicking on the Select background file or... button in the Background tab (see screenshot above). This will bring up a file navigator. Use the file navigator to locate and select the background image that you would like to use. If the image is correctly loaded in trackR, it should appear in the display window.

Note: the background image should have the same dimensions as the video.

2 - Creating a background image from the video

If you do not have recorded a background image or want to create one from the video, you first need to select the number of video frames you would like to use for estimating the background. You can do this using the Number of frames slider in the Background tab (see screenshot above). Selecting more images will usually lead to a more accurate result, but will take longer.

Once you have selected a number of frames, you need to select the type of algorithms that you would like to use for estimating the background image. Two algorithms are available in trackR:

  1. Mean: this calculates an average image from the selected video frames. This is a fast algorithm. However it does not always produce good results.
  2. Median: this calculates a median image from the selected video frames. This usually produces a better result than Mean, but will take a significantly longer amount of time.

You can select the algorithm using the Type dropdown menu. You can then click on the ...Estimate background button to start the computation of the background image.

Once the background image is estimated, it should appear in the display window. You can save this image for later use by clicking on the Save background file button.

3 - Removing ghosts

In some instance (e.g. when an object does not move for a significant portion of the video), the background image may contain “ghosts”. To remove a ghost, click on the Select ghost for removal button in the Background tab (see screenshot above). Then, use left-mouse clicks in the display window to draw a polygon around the ghost you want to remove (see image below for an example). Close the polygon using a right-mouse click. trackR will then use an inpainting algorithm to remove the ghost from the background image.

Repeat this operation as many times as necessary to remove all the unwanted ghosts.

Note: inpainting is not an exact science and the result of removing ghosts may not be satisfying. In this case, use a more advanced image editing program such as Adobe Photoshop or Gimp to clean up your background image.

2 - “Quantized” background subtraction

1 - Using an existing background image

If you have created and saved a background image following the instructions in the next section, you can load it by clicking on the Select background file or... button in the Background tab (see screenshot above). This will bring up a file navigator. Use the file navigator to locate and select the background image that you would like to use. If the image is correctly loaded in trackR, it should appear in the display window.

2 - Creating a background image from the video

If you want to create a background image from the video, you first need to select the number of video frames you would like to use for estimating the background. You can do this using the Number of frames slider in the Background tab (see screenshot above). Selecting more images will usually lead to a more accurate result, but will take longer.

Once you have selected a number of frames, you need to select the type of object that you would like to track. If the objects are generally darker than the background, select Objects darker than the background in the type dropdown menu. Otherwise, select Objects lighter than the background.

You can then click on the ...Estimate background button to start the computation of the background image.

Once the background image is estimated, it should appear in the display window. You can save this image for later use by clicking on the Save background file button.

3 - Removing ghosts

In some instance (e.g. when an object does not move for a significant portion of the video), the background image may contain “ghosts”. To remove a ghost, click on the Select ghost for removal button in the Background tab (see screenshot above). Then, use left-mouse clicks in the display window to draw a polygon around the ghost you want to remove (see image below for an example). Close the polygon using a right-mouse click. trackR will then use an inpainting algorithm to remove the ghost from the background image.

Repeat this operation as many times as necessary to remove all the unwanted ghosts.

Note: inpainting is not an exact science and the result of removing ghosts may not be satisfying. In this case, use a more advanced image editing program such as Adobe Photoshop or Gimp to clean up your background image.