Main idtrackerai objects¶

Video¶

Blob¶

class blob.Blob(centroid, contour, area, bounding_box_in_frame_coordinates, bounding_box_image=None, bounding_box_images_path=None, estimated_body_length=None, pixels=None, number_of_animals=None, frame_number=None, frame_number_in_video_path=None, in_frame_index=None, pixels_path=None, video_height=None, video_width=None, video_path=None, pixels_are_from_eroded_blob=False, resolution_reduction=1.0)[source]¶

Represents a segmented blob (collection of pixels) from a given frame.

A blob can represent a single animal or multiple animals during an occlusion or crossing.

Parameters

centroidtuple

Tuple (float, float) with the pixels coordinates of the blob center of mass.

contournumpy array

Array with the points that define the contour of the blob. The array is of the form [[[x1,y1]],[[x2,y2]],…,[[xn,yn]]].

areaint

Number of pixels that conform the blob.

bounding_box_in_frame_coordinateslist

Coordinates of the bounding box that encloses the segmented blob [(x, y), (x + bounding_box_width, y + bounding_box_height)]. Note that this bouding box is expanded some pixels with respect to the original bounding box obtained with OpenCV.

bounding_box_imagenumpy array or None, optional

Bonding box image that encloses the blob, by default None.

bounding_box_images_pathstr, optional

Path to the file where the bounding box images are stored, by default None.

estimated_body_lengthfloat, optional

Body length of the animal estimated from the diagonal of the original bounding box, by default None.

pixelslist, optional

List of pixels that define the blob, by default None.

number_of_animalsint, optional

Number of animals in the video as defined by the user, by default None.

frame_numberint, optional

Frame number in the video from which the blob was segmented, by default None.

frame_number_in_video_pathint, optional

Frame number in the video path from which the blob was segmented, by default None.

in_frame_indexint, optional

Index of the blob in the frame where it was segmented, by default None. This index comes from OpenCV and is defined by the hierarchy of the countours found in the frame.

pixels_pathstr, optional

Path to the file were the pixels of the blob are stored, by default None.

video_heightint, optional

Height of the video considering the resolution reduction factor, by default None.

video_width[type], optional

Width of the video considering the resolution reduction factor,: by default None.

video_pathstr, optional

Path to the video file from which the blob was segmented, by default None.

pixels_are_from_eroded_blobbool, optional

Flag to indicate if the pixels of the blobs come from from an eroded blob, by default False.

resolution_reductionfloat, optional

Resolution reductio factor as defined by the user, by default 1.0.

Attributes

accumulation_step: Integer indicating the accumulation step in which the blob
assigned_centroids: Centroids assigned to the blob during the tracking process.
assigned_identities: Identities assigned to the blob during the tracking process
blob_index: Blob index at the segmentation step (comes from the find contours
bounding_box_full_resolution: Bounding box cordinates without considering the resolution reduction factor, i.e.
bounding_box_image: Image cropped from the original video that contains the blob.
contour_full_resolution: Blob contour coordinates in pixels without considering the resolution reduction factor, i.e.
eroded_pixels: Pixels of the blob after erosion.
final_centroids: List of the animal/s centroid/s in the blob, considering the potential centroids that might have been aded by the user during the validation.
final_centroids_full_resolution: Returns the centroids considering the full resolution of the frame.
final_identities: Identities of the blob after the tracking process and after potential modifications by the users during the validation procedure.
fragment_identifier: Integer indicating the index of the fragment that contains the blob
identities_corrected_closing_gaps: Identity of the blob after crossings interpolation
identity: Identity of the blob assigned during the identification process
identity_corrected_solving_jumps: Identity of the blob after correcting impossible velocity jumps
is_a_crossing: Flag indicating whether the blob represents two or more animals together.
is_a_generated_blob: Flag indicating whether the blob was created by the user.
is_an_individual: Flag indicating whether the blob represent a single animal.
is_an_individual_in_a_fragment: Boolean indicating if the blob is in an individual fragment
is_in_a_fragment: Boolean indicating if the blob is in a fragment, i.e.
pixels: List of pixels that define the blob as determined from the segmentation parameters defined by the user.
pixels_set: A set() version of blob.pixels, used in blob.overlaps_with()
resolution_reduction: Resolution reduction factor defined by the user
summary: [Validation] Returns a summary string for some blob attributes.
used_for_training: Flag indicating if the blob has been used to train the
user_generated_centroids: List of centroids generated by the user during the validation processes.
user_generated_identities: List of identities of the blob some of which might have been give
was_a_crossing: Flag indicating whether the blob was created after splitting a

Methods

`add_centroid`(video, centroid, identity[, ...])	[Validation] Adds a centroid with a given identity to the blob.
`check_for_crossing_in_next_or_previous`([...])	Flag indicating if the blob has a crossing in its past or future overlapping history
`check_for_multiple_next_or_previous`([direction])	Flag indicating if the blob has multiple blobs in its past or future overlapping history
`delete_centroid`(video, identity, centroid, ...)	[Validation] Deletes a centroid of the blob.
`distance_from_countour_to`(point)	Returns the distance between point and the closest point in the contour of the blob.
`draw`(frame[, colors_lst, selected_id, ...])	[Validation] Draw the blob in a given frame of the video.
`get_image_for_identification`(...)	Gets the image used to train and evaluate the crossing detector CNN and the identification CNN.
`in_a_global_fragment_core`(blobs_in_frame)	Boolean indicating if a blob is in the core of a globalfragment.
`is_a_sure_crossing`()	Flag indicating that the blob is a sure crossing according to some heuristics and it can be used to train the crossing detector CNN.
`is_a_sure_individual`()	Flag indicating that the blob is a sure individual according to some heuristics and it can be used to train the crossing detector CNN.
`now_points_to`(other)	Given two consecutive blob objects updates their respective overlapping histories
`overlaps_with`(other)	Computes whether the pixels in self intersect with the pixels in other
`propagate_identity`(old_identity, ...)	[Validation] Propagates the new identity to next and previous blobs.
`removable_identity`(identity_to_remove, ...)	[Validation] Checks if the identity can be removed.
`save_image_for_identification`(...)	Saves in disk the image that will be used to train and evaluate the crossing detector CNN and the identification CNN.
`squared_distance_to`(other)	Returns the squared distance from the centroid of self to the centroid of other
`update_centroid`(video, old_centroid, ...)	[Validation] Updates the centroid of the blob.
`update_identity`(old_identity, new_identity, ...)	[Validation] Updates the identity of the blob.

property accumulation_step¶: Integer indicating the accumulation step in which the blob was assign by the cascade of training and identification protocols

add_centroid(video, centroid, identity, apply_resolution_reduction=True)[source]¶

[Validation] Adds a centroid with a given identity to the blob.

This method is used in the validation GUI. It is useful to add centroids for crossing blobs that are missing some centroids, or to individual blobs that should have been classified as crossings and are also missing some centroids.

Parameters

videoidtrackerai.video.Video: Instance of Video object
centroidtuple: Centroid to be added to the blob
identityint: Identity of the centroid to be added
apply_resolution_reductionbool, optional: Whether to consider the resolution reduction factor when adding the centroid, by default True. Note that the video is showed as full resolution in the validation GUI, but all centroids of the blobs consider the resolution reduction factor.

Raises

Exception: If the centroid is not a tuple of length 2
Exception: If the identity is not an integer between 1 and number of animals
Exception: If there is already another centroid with the same identity

property assigned_centroids¶

Centroids assigned to the blob during the tracking process.

It considers the default centroid of the blob at segmentation time or new centroids added to the blob during the interpolation of the crossings.

Returns

list: List of pairs (x, y) indicating the position of each individual in the blob.

property assigned_identities¶: Identities assigned to the blob during the tracking process

property blob_index¶: Blob index at the segmentation step (comes from the find contours function of OpenCV)

property bounding_box_full_resolution¶

Bounding box cordinates without considering the resolution reduction: factor, i.e. in the full resolution of the video.

Returns

numpy array, or None: Bounding box coordinates of the blob in full resolution of the video, [(x, y), (x + bounding_box_width, y + bounding_box_height)].

property bounding_box_image¶

Image cropped from the original video that contains the blob.

This image is used later to extract the image_for_identification that will be used to train and evaluate the crossing detector CNN and the identification CNN. This image can either be stored in the object (in RAM), or be stored in in a file (in DISK), or not be stored at all, in which case it is recomupted from the original video.

Returns

numpy array (uint8): Image cropped from the video containing the pixels that represent the blob.

check_for_crossing_in_next_or_previous(direction=None)[source]¶

Flag indicating if the blob has a crossing in its past or future overlapping history

This method is used to check whether the blob is an individual.

Parameters

directionstr: “previous” or “next”. If “previous” the past overlapping history will be checked in order to find out if the blob ends up in a crossing. Symmetrically, if “next” the future overlapping history of the blob will be checked.

Returns

bool: If True the blob has a crossing in its “past” or “future” history, depending on the parameter direction.

check_for_multiple_next_or_previous(direction=None)[source]¶

Flag indicating if the blob has multiple blobs in its past or future overlapping history

This method is used to check whether the blob is a crossing.

Parameters

directionstr: “previous” or “next”. If “previous” the past overlapping history will be checked in order to find out if the blob will split in the past. Symmetrically, if “next” the future overlapping history of the blob will be checked.

Returns

bool: If True the blob splits into two or multiple overlapping blobs in its “past” or “future” history, depending on the parameter “direction”.

property contour_full_resolution¶

Blob contour coordinates in pixels without considering the resolution reduction factor, i.e. in the full resolution of the video.

Returns

numpy array.: Coordinates in pixels of the points in the blob contour

delete_centroid(video, identity, centroid, blobs_in_frame, apply_resolution_reduction=True)[source]¶

[Validation] Deletes a centroid of the blob.

Parameters

videoidtrackerai.video.Video: Instance of Video object
identityint: Identity of the centroid to be deleted
centroidtuple: Centroid to be deleted from the blob
blobs_in_frameList: List of the blobs in the frame
apply_resolution_reductionbool, optional: Whether to consider the resolution reduction factor when adding the centroid, by default True. Note that the video is showed as full resolution in the validation GUI, but all centroids of the blobs consider the resolution reduction factor.

Raises

Exception: If the centroid is not a tuple of length 2
Exception: If the identity is unique in the frame
Exception: If it is the last centroid of the blob

distance_from_countour_to(point)[source]¶

Returns the distance between point and the closest point in the contour of the blob.

Parameters

pointtuple: (x,y)

Returns

float: Smallest distance between point and the contour of the blob.

draw(frame, colors_lst=None, selected_id=None, is_selected=False)[source]¶

[Validation] Draw the blob in a given frame of the video.

Parameters

framenumpy.array: Image where the blob should be draw.
colors_lst[type], optional: List of colors used to draw the blobs, by default None
selected_id[type], optional: Identity of the selected blob., by default None
is_selectedbool, optional: Flag indicated if the blob has been selected by the user, by default False

property eroded_pixels¶

Pixels of the blob after erosion.

The erosion is performed during the crossings interpolation process.

The pixels can either be stored in the object (in RAM), or be stored in a file (in DISK), or not be stored at all, in which case they are computed

Returns

List: List of integers indicating the linarized indices of the pixels that represent the blob after the erosion process.

property final_centroids¶

List of the animal/s centroid/s in the blob, considering the potential centroids that might have been aded by the user during the validation.

By default the centroid will be the center of mass of the blob of pixels defined by the blob. It can be different if the user modified the default centroid during validation or generated more centroids.

Returns

list: List of tuples (x, y) indicating the centroids of the blob.

property final_centroids_full_resolution¶

Returns the centroids considering the full resolution of the frame.

Note that after validation a blob can have more than one centroid. For example, this can happen in a crossing blob with missing centroids.

Returns

List of tuples (x, y), where x, and y are coordinates in the full
resolution of the video frame.

property final_identities¶: Identities of the blob after the tracking process and after potential modifications by the users during the validation procedure.

Table of Contents

Main idtrackerai objects¶

Video¶

Blob¶

List of blobs¶

Fragment¶

List of fragments¶

Global fragment¶

List of global fragments¶