Main idtrackerai objects¶
Video¶
Blob¶
- class blob.Blob(centroid, contour, area, bounding_box_in_frame_coordinates, bounding_box_image=None, bounding_box_images_path=None, estimated_body_length=None, pixels=None, number_of_animals=None, frame_number=None, frame_number_in_video_path=None, in_frame_index=None, pixels_path=None, video_height=None, video_width=None, video_path=None, pixels_are_from_eroded_blob=False, resolution_reduction=1.0)[source]¶
Represents a segmented blob (collection of pixels) from a given frame.
A blob can represent a single animal or multiple animals during an occlusion or crossing.
- Parameters
- centroidtuple
Tuple (float, float) with the pixels coordinates of the blob center of mass.
- contournumpy array
Array with the points that define the contour of the blob. The array is of the form [[[x1,y1]],[[x2,y2]],…,[[xn,yn]]].
- areaint
Number of pixels that conform the blob.
- bounding_box_in_frame_coordinateslist
Coordinates of the bounding box that encloses the segmented blob [(x, y), (x + bounding_box_width, y + bounding_box_height)]. Note that this bouding box is expanded some pixels with respect to the original bounding box obtained with OpenCV.
- bounding_box_imagenumpy array or None, optional
Bonding box image that encloses the blob, by default None.
- bounding_box_images_pathstr, optional
Path to the file where the bounding box images are stored, by default None.
- estimated_body_lengthfloat, optional
Body length of the animal estimated from the diagonal of the original bounding box, by default None.
- pixelslist, optional
List of pixels that define the blob, by default None.
- number_of_animalsint, optional
Number of animals in the video as defined by the user, by default None.
- frame_numberint, optional
Frame number in the video from which the blob was segmented, by default None.
- frame_number_in_video_pathint, optional
Frame number in the video path from which the blob was segmented, by default None.
- in_frame_indexint, optional
Index of the blob in the frame where it was segmented, by default None. This index comes from OpenCV and is defined by the hierarchy of the countours found in the frame.
- pixels_pathstr, optional
Path to the file were the pixels of the blob are stored, by default None.
- video_heightint, optional
Height of the video considering the resolution reduction factor, by default None.
- video_width[type], optional
- Width of the video considering the resolution reduction factor,
by default None.
- video_pathstr, optional
Path to the video file from which the blob was segmented, by default None.
- pixels_are_from_eroded_blobbool, optional
Flag to indicate if the pixels of the blobs come from from an eroded blob, by default False.
- resolution_reductionfloat, optional
Resolution reductio factor as defined by the user, by default 1.0.
- Attributes
accumulation_step
Integer indicating the accumulation step in which the blob
assigned_centroids
Centroids assigned to the blob during the tracking process.
assigned_identities
Identities assigned to the blob during the tracking process
blob_index
Blob index at the segmentation step (comes from the find contours
bounding_box_full_resolution
Bounding box cordinates without considering the resolution reduction factor, i.e.
bounding_box_image
Image cropped from the original video that contains the blob.
contour_full_resolution
Blob contour coordinates in pixels without considering the resolution reduction factor, i.e.
eroded_pixels
Pixels of the blob after erosion.
final_centroids
List of the animal/s centroid/s in the blob, considering the potential centroids that might have been aded by the user during the validation.
final_centroids_full_resolution
Returns the centroids considering the full resolution of the frame.
final_identities
Identities of the blob after the tracking process and after potential modifications by the users during the validation procedure.
fragment_identifier
Integer indicating the index of the fragment that contains the blob
identities_corrected_closing_gaps
Identity of the blob after crossings interpolation
identity
Identity of the blob assigned during the identification process
identity_corrected_solving_jumps
Identity of the blob after correcting impossible velocity jumps
is_a_crossing
Flag indicating whether the blob represents two or more animals together.
is_a_generated_blob
Flag indicating whether the blob was created by the user.
is_an_individual
Flag indicating whether the blob represent a single animal.
is_an_individual_in_a_fragment
Boolean indicating if the blob is in an individual fragment
is_in_a_fragment
Boolean indicating if the blob is in a fragment, i.e.
pixels
List of pixels that define the blob as determined from the segmentation parameters defined by the user.
resolution_reduction
Resolution reduction factor defined by the user
summary
[Validation] Returns a summary string for some blob attributes.
used_for_training
Flag indicating if the blob has been used to train the
user_generated_centroids
List of centroids generated by the user during the validation processes.
user_generated_identities
List of identities of the blob some of which might have been give
was_a_crossing
Flag indicating whether the blob was created after splitting a
Methods
add_centroid
(video, centroid, identity[, ...])[Validation] Adds a centroid with a given identity to the blob.
Flag indicating if the blob has a crossing in its past or future overlapping history
check_for_multiple_next_or_previous
([direction])Flag indicating if the blob has multiple blobs in its past or future overlapping history
delete_centroid
(video, identity, centroid, ...)[Validation] Deletes a centroid of the blob.
distance_from_countour_to
(point)Returns the distance between point and the closest point in the contour of the blob.
draw
(frame[, colors_lst, selected_id, ...])[Validation] Draw the blob in a given frame of the video.
Gets the image used to train and evaluate the crossing detector CNN and the identification CNN.
in_a_global_fragment_core
(blobs_in_frame)Boolean indicating if a blob is in the core of a globalfragment.
Flag indicating that the blob is a sure crossing according to some heuristics and it can be used to train the crossing detector CNN.
Flag indicating that the blob is a sure individual according to some heuristics and it can be used to train the crossing detector CNN.
now_points_to
(other)Given two consecutive blob objects updates their respective overlapping histories
overlaps_with
(other)Computes whether the pixels in self intersect with the pixels in other
propagate_identity
(old_identity, ...)[Validation] Propagates the new identity to next and previous blobs.
removable_identity
(identity_to_remove, ...)[Validation] Checks if the identity can be removed.
Saves in disk the image that will be used to train and evaluate the crossing detector CNN and the identification CNN.
squared_distance_to
(other)Returns the squared distance from the centroid of self to the centroid of other
update_centroid
(video, old_centroid, ...)[Validation] Updates the centroid of the blob.
update_identity
(old_identity, new_identity, ...)[Validation] Updates the identity of the blob.
- property accumulation_step¶
Integer indicating the accumulation step in which the blob was assign by the cascade of training and identification protocols
- add_centroid(video, centroid, identity, apply_resolution_reduction=True)[source]¶
[Validation] Adds a centroid with a given identity to the blob.
This method is used in the validation GUI. It is useful to add centroids for crossing blobs that are missing some centroids, or to individual blobs that should have been classified as crossings and are also missing some centroids.
- Parameters
- videoidtrackerai.video.Video
Instance of Video object
- centroidtuple
Centroid to be added to the blob
- identityint
Identity of the centroid to be added
- apply_resolution_reductionbool, optional
Whether to consider the resolution reduction factor when adding the centroid, by default True. Note that the video is showed as full resolution in the validation GUI, but all centroids of the blobs consider the resolution reduction factor.
- Raises
- Exception
If the centroid is not a tuple of length 2
- Exception
If the identity is not an integer between 1 and number of animals
- Exception
If there is already another centroid with the same identity
- property assigned_centroids¶
Centroids assigned to the blob during the tracking process.
It considers the default centroid of the blob at segmentation time or new centroids added to the blob during the interpolation of the crossings.
- Returns
- list
List of pairs (x, y) indicating the position of each individual in the blob.
- property assigned_identities¶
Identities assigned to the blob during the tracking process
- property blob_index¶
Blob index at the segmentation step (comes from the find contours function of OpenCV)
- property bounding_box_full_resolution¶
- Bounding box cordinates without considering the resolution reduction
factor, i.e. in the full resolution of the video.
- Returns
- numpy array, or None
Bounding box coordinates of the blob in full resolution of the video, [(x, y), (x + bounding_box_width, y + bounding_box_height)].
- property bounding_box_image¶
Image cropped from the original video that contains the blob.
This image is used later to extract the image_for_identification that will be used to train and evaluate the crossing detector CNN and the identification CNN. This image can either be stored in the object (in RAM), or be stored in in a file (in DISK), or not be stored at all, in which case it is recomupted from the original video.
- Returns
- numpy array (uint8)
Image cropped from the video containing the pixels that represent the blob.
- check_for_crossing_in_next_or_previous(direction=None)[source]¶
Flag indicating if the blob has a crossing in its past or future overlapping history
This method is used to check whether the blob is an individual.
- Parameters
- directionstr
“previous” or “next”. If “previous” the past overlapping history will be checked in order to find out if the blob ends up in a crossing. Symmetrically, if “next” the future overlapping history of the blob will be checked.
- Returns
- bool
If True the blob has a crossing in its “past” or “future” history, depending on the parameter direction.
- check_for_multiple_next_or_previous(direction=None)[source]¶
Flag indicating if the blob has multiple blobs in its past or future overlapping history
This method is used to check whether the blob is a crossing.
- Parameters
- directionstr
“previous” or “next”. If “previous” the past overlapping history will be checked in order to find out if the blob will split in the past. Symmetrically, if “next” the future overlapping history of the blob will be checked.
- Returns
- bool
If True the blob splits into two or multiple overlapping blobs in its “past” or “future” history, depending on the parameter “direction”.
- property contour_full_resolution¶
Blob contour coordinates in pixels without considering the resolution reduction factor, i.e. in the full resolution of the video.
- Returns
- numpy array.
Coordinates in pixels of the points in the blob contour
- delete_centroid(video, identity, centroid, blobs_in_frame, apply_resolution_reduction=True)[source]¶
[Validation] Deletes a centroid of the blob.
- Parameters
- videoidtrackerai.video.Video
Instance of Video object
- identityint
Identity of the centroid to be deleted
- centroidtuple
Centroid to be deleted from the blob
- blobs_in_frameList
List of the blobs in the frame
- apply_resolution_reductionbool, optional
Whether to consider the resolution reduction factor when adding the centroid, by default True. Note that the video is showed as full resolution in the validation GUI, but all centroids of the blobs consider the resolution reduction factor.
- Raises
- Exception
If the centroid is not a tuple of length 2
- Exception
If the identity is unique in the frame
- Exception
If it is the last centroid of the blob
- distance_from_countour_to(point)[source]¶
Returns the distance between point and the closest point in the contour of the blob.
- Parameters
- pointtuple
(x,y)
- Returns
- float
Smallest distance between point and the contour of the blob.
- draw(frame, colors_lst=None, selected_id=None, is_selected=False)[source]¶
[Validation] Draw the blob in a given frame of the video.
- Parameters
- framenumpy.array
Image where the blob should be draw.
- colors_lst[type], optional
List of colors used to draw the blobs, by default None
- selected_id[type], optional
Identity of the selected blob., by default None
- is_selectedbool, optional
Flag indicated if the blob has been selected by the user, by default False
- property eroded_pixels¶
Pixels of the blob after erosion.
The erosion is performed during the crossings interpolation process.
The pixels can either be stored in the object (in RAM), or be stored in a file (in DISK), or not be stored at all, in which case they are computed
- Returns
- List
List of integers indicating the linarized indices of the pixels that represent the blob after the erosion process.
- property final_centroids¶
List of the animal/s centroid/s in the blob, considering the potential centroids that might have been aded by the user during the validation.
By default the centroid will be the center of mass of the blob of pixels defined by the blob. It can be different if the user modified the default centroid during validation or generated more centroids.
- Returns
- list
List of tuples (x, y) indicating the centroids of the blob.
- property final_centroids_full_resolution¶
Returns the centroids considering the full resolution of the frame.
Note that after validation a blob can have more than one centroid. For example, this can happen in a crossing blob with missing centroids.
- Returns
- List of tuples (x, y), where x, and y are coordinates in the full
- resolution of the video frame.
- property final_identities¶
Identities of the blob after the tracking process and after potential modifications by the users during the validation procedure.
- property fragment_identifier¶
Integer indicating the index of the fragment that contains the blob
See also
Fragment
- get_image_for_identification(identification_image_size, height, width)[source]¶
Gets the image used to train and evaluate the crossing detector CNN and the identification CNN.
- Parameters
- identification_image_sizetuple
Dimensions of the identification image (height, widht, channels). Channels is always 1 as images in color are still not considered.
- heightint
Video height considering resolution reduction factor.
- widthint
Video width considering resolution reduction factor.
- Returns
- ndarray
Square image with black background used to train the crossings detector CNN and the identifiactio CNN.
- property identities_corrected_closing_gaps¶
Identity of the blob after crossings interpolation
- property identity¶
Identity of the blob assigned during the identification process
- property identity_corrected_solving_jumps¶
Identity of the blob after correcting impossible velocity jumps
- in_a_global_fragment_core(blobs_in_frame)[source]¶
Boolean indicating if a blob is in the core of a globalfragment.
A blob in a frame is in the core of a global fragment if in that frame there are as many blobs as number of animals to track
- Parameters
- blobs_in_framelist
List of Blob objects representing the animals segmented in the frame self.frame_number
- Returns
- bool
True if the blob is in the core of a global fragment
- property is_a_crossing¶
Flag indicating whether the blob represents two or more animals together.
This attribute is the negative of is_an_individual and is set at the same time as is an individual
- property is_a_generated_blob¶
Flag indicating whether the blob was created by the user.
Blobs created by the users during the validation process do not have a contour as only the centroid is created
- is_a_sure_crossing()[source]¶
Flag indicating that the blob is a sure crossing according to some heuristics and it can be used to train the crossing detector CNN.
- Returns
- bool
- is_a_sure_individual()[source]¶
Flag indicating that the blob is a sure individual according to some heuristics and it can be used to train the crossing detector CNN.
- Returns
- bool
- property is_an_individual¶
Flag indicating whether the blob represent a single animal.
The private attribute _is_an_individual is assigned during the crossings detection process.
- property is_an_individual_in_a_fragment¶
Boolean indicating if the blob is in an individual fragment
- property is_in_a_fragment¶
Boolean indicating if the blob is in a fragment, i.e. is not an isolated blob or is in the extreme of a fragment
- now_points_to(other)[source]¶
Given two consecutive blob objects updates their respective overlapping histories
- Parameters
- other<Blob object>
An instance of the class Blob
- overlaps_with(other)[source]¶
Computes whether the pixels in self intersect with the pixels in other
- Parameters
- other<Blob object>
An instance of the class Blob
- Returns
- bool
True if the lists of pixels of both blobs have non-empty intersection
- property pixels¶
List of pixels that define the blob as determined from the segmentation parameters defined by the user.
Note that pixels are represented as linearized coordinates of the frame (considering the resolution reduction factor), i.e. each pixel is a single integer.
The pixels can either be stored in the object (in RAM), or be stored in a file (in DISK), or not be stored at all, in which case they are recomputed from the contour and the original frame of the video.
- Returns
- List
List of integers indicating the linarized indices of the pixels that represent the blob.
- propagate_identity(old_identity, new_identity, centroid)[source]¶
[Validation] Propagates the new identity to next and previous blobs.
This method called in the validation GUI when the used updates the identity of a given blob.
- Parameters
- old_identityint
Previous identity of the blob
- new_identityint
New ientity of the blob
- centroidtuple
[description]
- removable_identity(identity_to_remove, blobs_in_frame)[source]¶
[Validation] Checks if the identity can be removed.
- Parameters
- identity_to_removeint
Identity to be removed
- blobs_in_framelist
List of Blob objects in the frame where the identity is going to be removed
- Returns
- bool
True if the identity can be removed.
- property resolution_reduction¶
Resolution reduction factor defined by the user
- save_image_for_identification(identification_image_size, height, width, file_path)[source]¶
Saves in disk the image that will be used to train and evaluate the crossing detector CNN and the identification CNN.
This also updates the identification_image_index and the episode attributes. This helps to load the image from the correct file_path.
- Parameters
- identification_image_sizetuple
Tuple of integers (height, width, channels).
- heightint
Video height considering the resolution reduction factor.
- widthint
Video width considering the resolution reduction factor.
- file_pathstr
Path to the hdf5 file where the images will be stored.
- squared_distance_to(other)[source]¶
Returns the squared distance from the centroid of self to the centroid of other
- Parameters
- other<Blob object> or tuple
An instance of the class Blob or a tuple (x,y)
- Returns
- float
Squared distance between centroids
- property summary¶
[Validation] Returns a summary string for some blob attributes.
- Returns
- str
Summary description of the blob
- update_centroid(video, old_centroid, new_centroid, identity)[source]¶
[Validation] Updates the centroid of the blob.
- Parameters
- videoidtrackerai.video.Video
Instance of Video object
- old_centroidtuple
Centroid to be updated
- new_centroidtuple
Coordinates of the new centroid
- identityint
Identity of the centroid to be updated
- update_identity(old_identity, new_identity, centroid)[source]¶
[Validation] Updates the identity of the blob.
This method is used during the validation GUI. It populates the private attributes _user_generated_identities and _user_generated_centroids.
- Parameters
- new_identityint
new value for the identity of the blob
- old_identityint
old value of the identity of the blob. It must be specified when the blob has multiple identities already assigned.
- centroidtuple
centroid which identity must be updated.
- property used_for_training¶
Flag indicating if the blob has been used to train the identification CNN
- property user_generated_centroids¶
List of centroids generated by the user during the validation processes.
- Returns
- list or None
If the user has not generated any centroid for this blob, it returns None.
- property user_generated_identities¶
List of identities of the blob some of which might have been give by a user during the validation process
- property was_a_crossing¶
Flag indicating whether the blob was created after splitting a crossing blob during the crossings interpolation process