B.Tech - Computer Science / Information Science / Information Technology - Semester V Advanced Python A Comprehensive Undergraduate Textbook CHAPTER 2 Video Processing and Object Tracking Using OpenCV SCHOOL OF ENGINEERING / COMPUTER SCIENCE DEPARTMENT OF COMPUTER SCIENCE / INFORMATION SCIENCE / INFORMATION TECHNOLOGY Chapter Introduction / Overview Videos are one of the most important sources of visual information in modern computer vision. A video is not handled as one continuous image; it is processed as a time-ordered sequence of frames. OpenCV makes this process practical by providing tools to read video files, capture live video from cameras, access individual frames, draw information on frames, save processed videos, detect motion, and track objects across time. This chapter introduces the basic and intermediate video-processing operations required for real-time computer vision applications. Students first learn how video files are represented, how containers and codecs affect reading and writing, and how OpenCV converts a video stream into frames. The chapter then explains frame extraction, camera capture, video display, video writing, annotations such as lines and rectangles, timestamp overlay, video property control, motion detection, and object tracking. By the end of this chapter, students should be able to build small video-analysis programs that read input from files or cameras, process every frame in a loop, visually annotate the output, and save or display the result in real time. Figure 2.1: Typical OpenCV video processing pipeline from input source to processed output Learning Outcomes Upon successful completion of this chapter, students will be able to: 1. Explain how videos are represented as sequences of frames and differentiate between containers, codecs, frame rate and resolution. 2. Use OpenCV VideoCapture to open a video file and extract frames at required intervals. 3. Capture live video from a camera and handle basic camera-access errors. 4. Use VideoWriter to save processed frames as a video file with a suitable codec, frame size and FPS. 5. Draw lines, rectangles and circles on frames and overlay text such as date and time. 6. Read and set common video properties such as frame width, height, FPS, frame count and position. 7. Implement simple motion detection using frame differencing and background subtraction. 8. Explain the difference between motion detection, object detection and object tracking. 9. Use OpenCV trackers such as MIL, KCF and CSRT for single-object tracking. 10. Identify common errors in video processing and apply practical troubleshooting methods. Key Concepts Glossary of Key Terms in This Chapter Frame: A single image in a video sequence. Video processing in OpenCV usually works by reading and processing one frame at a time. Frame Rate / FPS: The number of frames displayed or recorded per second. Common values include 24, 25, 30 and 60 FPS. Resolution: The width and height of each video frame, usually expressed as pixels such as 640 × 480 or 1920 × 1080. Container: The file wrapper that stores video, audio and metadata. Examples include MP4, AVI, MKV and MOV. Codec: The compression/decompression method used to encode video data. Examples include H.264, MJPEG and XVID. VideoCapture: OpenCV class used to read video files, image sequences, cameras or streams. VideoWriter: OpenCV class used to encode and save frames into a video file. FourCC: A four-character code that identifies the video codec used by VideoWriter, such as MJPG or XVID. Foreground Mask: A binary image produced by motion-detection algorithms where moving pixels are separated from the static background. Object Tracking: The process of estimating the location of a selected object across successive video frames. 2.1 Understanding Video Files and Formats in OpenCV A digital video is formed by displaying many still images, called frames, in rapid succession. OpenCV reads a video by decoding it into frames. Each frame is represented internally as an image matrix, so most image-processing operations that work on images can also be applied to videos by looping through frames. Figure 2.2: Components of a video file used during OpenCV reading and writing 2.1.1 Container, Codec, FPS and Resolution The file extension alone does not fully define a video. A video file consists of a container, one or more encoded streams, and metadata. The container decides how data is stored, while the codec decides how frames are compressed. When writing videos with OpenCV, the codec, frame size and FPS must be selected carefully because mismatched settings can produce unreadable or empty files. Table 2.1: Common video file formats and their practical use in OpenCV Format / Term Meaning Examples / Notes MP4 Popular container format for compressed video. Often uses H.264/H.265 codecs; widely supported. AVI Older container format commonly used in OpenCV examples. Works well with MJPG or XVID codecs for classroom practice. MKV Flexible container that can hold many codecs. Useful for high-quality storage but codec support depends on system backend. MOV QuickTime container format. Common in cameras and mobile devices. FourCC Four-character codec identifier. Example: cv.VideoWriter_fourcc(*"XVID"). FPS Number of frames per second. Controls playback speed and timing. Resolution Frame width and height in pixels. Must match the actual frames being written. Did You Know?: OpenCV usually uses BGR channel order for color images and frames, not RGB. Therefore, a color tuple such as (255, 0, 0) appears blue in OpenCV, while (0, 0, 255) appears red. 2.1.2 How OpenCV Treats a Video OpenCV does not normally process the complete video at once. Instead, it opens the video source, reads one frame, processes that frame, displays or writes the output, and then repeats the loop until the video ends or the user stops the program. This approach makes real-time applications possible even for long videos. Table 2.2: Basic workflow for video processing in OpenCV Step OpenCV Operation Purpose 1 cap = cv.VideoCapture(source) Open a file, camera or stream. 2 ret, frame = cap.read() Read the next frame. 3 Process frame Apply resize, drawing, detection or tracking. 4 cv.imshow() Display the frame or result. 5 writer.write(frame) Optional: save the processed frame. 6 cap.release(); cv.destroyAllWindows() Release resources safely. 2.2 Extracting Frames from Videos Frame extraction means saving selected frames from a video as separate image files. It is useful for dataset preparation, manual annotation, debugging, creating thumbnails, training computer vision models, and studying motion at selected time intervals. 2.2.1 Extracting Every Frame The simplest method is to read every frame in a loop and save it using cv.imwrite(). A frame counter is used to create unique file names. i mport cv2 as cv import os video_path = "input_video.mp4" output_dir = "frames" os.makedirs(output_dir, exist_ok=True) cap = cv.VideoCapture(video_path) frame_no = 0 while True: ret, frame = cap.read() if not ret: break filename = os.path.join(output_dir, f"frame_{frame_no:05d}.jpg") cv.imwrite(filename, frame) frame_no += 1 cap.release() print("Total frames extracted:", frame_no) 2.2.2 Extracting Frames at Intervals Saving every frame can create thousands of images. To reduce storage, frames can be saved at fixed intervals, such as every 10th frame or every 1 second. Interval-based extraction is useful when only representative frames are needed. import cv2 as cv import os cap = cv.VideoCapture("input_video.mp4") fps = cap.get(cv.CAP_PROP_FPS) interval = int(fps) # approximately one frame per second count = 0 saved = 0 os.makedirs("sampled_frames", exist_ok=True) while True: ret, frame = cap.read() if not ret: break if count % interval == 0: cv.imwrite(f"sampled_frames/second_{saved:04d}.jpg", frame) saved += 1 count += 1 cap.release() Table 2.3: Frame extraction use cases Use Case Why Frames Are Extracted Example Dataset creation To create individual images for labeling and training. Traffic frames for vehicle classification. Debugging To inspect frame quality and processing errors. Checking blur or low-light frames. Event analysis To study important moments in a video. Saving frames when motion is detected. Thumbnail generation To represent video content with a still image. Preview image for a lecture video. Annotation To mark object boundaries manually. Drawing boxes around people or vehicles. Practical Note: Always check whether cap.isOpened() is True before reading. If the path is wrong, the codec is unsupported, or permission is denied, OpenCV may fail to read frames. 2.3 Reading Videos from Files Reading a video file is the most common starting point in video processing. OpenCV uses VideoCapture to open a file path. Each call to read() returns a Boolean value and the next frame. The Boolean value is True when the frame is read successfully and False when the video ends or an error occurs. 2.3.1 Basic Program to Read and Display a Video import cv2 as cv cap = cv.VideoCapture("input_video.mp4") if not cap.isOpened(): print("Error: Cannot open video file") exit() while True: ret, frame = cap.read() if not ret: break cv.imshow("Video", frame) # Press q to stop playback if cv.waitKey(25) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() 2.3.2 Important VideoCapture Methods Table 2.4: Common VideoCapture methods and their purpose Method Purpose Example VideoCapture(source) Opens a video file, camera index or stream URL. cv.VideoCapture("video.mp4") isOpened() Checks whether the source opened successfully. cap.isOpened() read() Grabs, decodes and returns the next frame. ret, frame = cap.read() get(propId) Reads a video property. cap.get(cv.CAP_PROP_FPS) set(propId, value) Attempts to set a property. cap.set(cv.CAP_PROP_FRAME_WIDTH, 640) release() Releases the file or camera resource. cap.release() Key Insight: Video playback speed depends on waitKey delay. A smaller delay gives faster playback, while a larger delay slows it down. For a 30 FPS video, waitKey(33) is approximately real-time. 2.3.3 Seeking to a Specific Frame or Time For video files, OpenCV properties allow moving to a specific frame number or time position. This is useful when only a particular segment of a video must be analysed. import cv2 as cv cap = cv.VideoCapture("input_video.mp4") # Move to the 500th frame cap.set(cv.CAP_PROP_POS_FRAMES, 500) ret, frame = cap.read() if ret: cv.imshow("Frame 500", frame) cv.waitKey(0) cap.release() cv.destroyAllWindows() 2.4 Capturing Videos from Camera Camera capture is used for real-time computer vision applications such as face detection, attendance systems, gesture recognition, surveillance, robotics and laboratory automation. In OpenCV, the default camera is usually opened using device index 0. External cameras may use index 1, 2 or higher depending on the operating system. 2.4.1 Basic Webcam Capture Program import cv2 as cv cap = cv.VideoCapture(0) # 0 means default camera if not cap.isOpened(): print("Error: Cannot access camera") exit() while True: ret, frame = cap.read() if not ret: print("Error: Cannot receive frame") break cv.imshow("Camera Feed", frame) if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() Table 2.5: Camera capture problems and solutions Problem Possible Cause Solution Camera not opening Wrong device index or camera used by another app. Try VideoCapture(1), close other camera apps, check permissions. Blank frame Camera disconnected or driver issue. Reconnect camera, restart program, check OS camera settings. Low FPS High resolution or poor lighting. Reduce resolution, improve lighting, use faster camera. Property not changing Camera/backend does not support requested setting. Check return value of cap.set() and use supported values. Delayed display Processing is slower than capture speed. Resize frames, simplify processing, skip frames, use threading. 2.4.2 Capturing from IP Camera or Stream If an IP camera or network stream is available, VideoCapture can also open a stream URL. The exact URL depends on the camera model and protocol. Network delay, authentication and codec support can affect performance. import cv2 as cv stream_url = "rtsp://username:password@192.168.1.10:554/stream1" cap = cv.VideoCapture(stream_url) while cap.isOpened(): ret, frame = cap.read() if not ret: break cv.imshow("IP Camera", frame) if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() 2.5 Writing and Displaying Videos After processing a video, the output may need to be saved for presentation, analysis or evidence. OpenCV writes video using VideoWriter. To create a VideoWriter object, we need an output file name, codec, frame rate and frame size. Every processed frame must have the same size as the writer configuration. 2.5.1 VideoWriter Parameters Table 2.6: Parameters required for writing videos Parameter Meaning Example filename Name of the output video file. "output.avi" fourcc Four-character codec code. cv.VideoWriter_fourcc(*"XVID") fps Frames per second of output video. 20.0 or 30.0 frameSize Width and height of output frames. (640, 480) isColor Whether the output is color or grayscale. True for color frames 2.5.2 Writing Processed Camera Video import cv2 as cv cap = cv.VideoCapture(0) width = int(cap.get(cv.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv.CAP_PROP_FRAME_HEIGHT)) fps = 20.0 fourcc = cv.VideoWriter_fourcc(*"XVID") out = cv.VideoWriter("camera_output.avi", fourcc, fps, (width, height)) while True: ret, frame = cap.read() if not ret: break cv.imshow("Recording", frame) out.write(frame) if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() out.release() cv.destroyAllWindows() 2.5.3 Displaying Multiple Windows OpenCV can display multiple windows at the same time. For example, one window can show the original frame and another can show a grayscale or motion mask output. This is useful for comparing processing stages. import cv2 as cv cap = cv.VideoCapture("input_video.mp4") while True: ret, frame = cap.read() if not ret: break gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY) cv.imshow("Original", frame) cv.imshow("Grayscale", gray) if cv.waitKey(25) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() Important: When writing a video, the frame size passed to VideoWriter must match the actual size of every frame written. If frames are resized during processing, create the writer using the resized width and height. 2.6 Drawing Lines, Rectangles, Circles and Text on Videos Drawing operations are applied to each frame before it is displayed or written. This allows us to create visual annotations such as object boundaries, tracking boxes, warning zones, counters, timestamps, labels and measurement guides. 2.6.1 Common Drawing Functions Table 2.7: Drawing functions used on video frames Function Purpose Basic Syntax cv.line() Draws a straight line between two points. cv.line(frame, pt1, pt2, color, thickness) cv.rectangle() Draws a rectangle using two corner points. cv.rectangle(frame, pt1, pt2, color, thickness) cv.circle() Draws a circle using center and radius. cv.circle(frame, center, radius, color, thickness) cv.putText() Writes text on the frame. cv.putText(frame, text, org, font, scale, color, thickness) cv.polylines() Draws connected line segments. cv.polylines(frame, pts, isClosed, color) 2.6.2 Drawing Shapes on Live Video import cv2 as cv cap = cv.VideoCapture(0) while True: ret, frame = cap.read() if not ret: break h, w = frame.shape[:2] cv.line(frame, (0, h//2), (w, h//2), (255, 0, 0), 2) cv.rectangle(frame, (50, 50), (250, 180), (0, 255, 0), 2) cv.circle(frame, (w//2, h//2), 60, (0, 0, 255), 3) cv.imshow("Drawing on Video", frame) if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() 2.6.3 Adding Text with Date and Time Timestamps are important in surveillance, experiments and record keeping. Python’s datetime module can generate date and time text, and OpenCV can write it on every frame using cv.putText(). import cv2 as cv from datetime import datetime cap = cv.VideoCapture(0) font = cv.FONT_HERSHEY_SIMPLEX while True: ret, frame = cap.read() if not ret: break timestamp = datetime.now().strftime("%d-%m-%Y %H:%M:%S") cv.rectangle(frame, (10, 10), (360, 50), (0, 0, 0), -1) cv.putText(frame, timestamp, (20, 40), font, 0.7, (255, 255, 255), 2, cv.LINE_AA) cv.imshow("Timestamped Video", frame) if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() Formatting Tip: Use cv.LINE_AA for smoother text and curves. Use a filled rectangle behind text when the background is complex so that the label remains readable. 2.7 Setting Video Properties Video properties describe the current video source and its frames. They include frame width, frame height, FPS, frame count, codec information, brightness, contrast and current frame position. OpenCV properties are accessed using cap.get() and cap.set(). Some properties can be read from both video files and cameras, while others are only meaningful for cameras. 2.7.1 Common Video Properties Table 2.8: Common video properties in OpenCV Property Meaning Typical Use CAP_PROP_FRAME_WIDTH Width of frames in pixels. Read or set camera resolution. CAP_PROP_FRAME_HEIGHT Height of frames in pixels. Read or set camera resolution. CAP_PROP_FPS Frame rate of video source. Use FPS for playback and writing. Property Meaning Typical Use CAP_PROP_FRAME_COUNT Total number of frames in a video file. Estimate duration or progress. CAP_PROP_POS_FRAMES Current frame position. Jump to a frame number. CAP_PROP_POS_MSEC Current position in milliseconds. Seek to a time location. CAP_PROP_FOURCC Codec code used by the video. Inspect file codec. CAP_PROP_BRIGHTNESS Camera brightness setting. Adjust camera if supported. CAP_PROP_CONTRAST Camera contrast setting. Adjust camera if supported. 2.7.2 Reading Video Properties 2.7.3 Setting Camera Resolution import cv2 as cv cap = cv.VideoCapture(0) cap.set(cv.CAP_PROP_FRAME_WIDTH, 1280) cap.set(cv.CAP_PROP_FRAME_HEIGHT, 720) cap.set(cv.CAP_PROP_FPS, 30) actual_width = cap.get(cv.CAP_PROP_FRAME_WIDTH) actual_height = cap.get(cv.CAP_PROP_FRAME_HEIGHT) actual_fps = cap.get(cv.CAP_PROP_FPS) print("Actual settings:", actual_width, actual_height, actual_fps) cap.release() Important: import cv2 as cv cap = cv.VideoCapture("input_video.mp4") width = cap.get(cv.CAP_PROP_FRAME_WIDTH) height = cap.get(cv.CAP_PROP_FRAME_HEIGHT) fps = cap.get(cv.CAP_PROP_FPS) frames = cap.get(cv.CAP_PROP_FRAME_COUNT) duration = frames / fps if fps > 0 else 0 print("Width:", width) print("Height:", height) print("FPS:", fps) print("Frame count:", frames) print("Duration in seconds:", duration) cap.release() cap.set() is a request, not a guarantee. The camera, driver and backend decide whether the requested resolution or FPS is supported. Always read the property again using cap.get() to confirm the actual value. 2.8 Motion Detection Using OpenCV Motion detection is the process of identifying parts of a video frame that have changed over time. It is different from object tracking. Motion detection answers the question “Where is movement happening?”, while tracking answers “Where is this selected object now?”. Figure 2.3: Motion detection workflow using background subtraction 2.8.1 Frame Differencing Frame differencing compares two consecutive frames or compares the current frame with a reference frame. Pixels that change significantly are treated as motion. This method is simple and fast but is sensitive to camera movement and lighting changes. import cv2 as cv cap = cv.VideoCapture(0) ret, prev = cap.read() prev_gray = cv.cvtColor(prev, cv.COLOR_BGR2GRAY) prev_gray = cv.GaussianBlur(prev_gray, (21, 21), 0) while True: ret, frame = cap.read() if not ret: break gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY) gray = cv.GaussianBlur(gray, (21, 21), 0) diff = cv.absdiff(prev_gray, gray) _, thresh = cv.threshold(diff, 25, 255, cv.THRESH_BINARY) thresh = cv.dilate(thresh, None, iterations=2) cv.imshow("Motion Mask", thresh) cv.imshow("Frame", frame) prev_gray = gray if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() 2.8.2 Background Subtraction Background subtraction builds a model of the static background and subtracts it from the current frame. Moving pixels become part of the foreground mask. OpenCV provides commonly used background subtractors such as MOG2 and KNN. These methods are useful for stationary cameras such as surveillance cameras. import cv2 as cv cap = cv.VideoCapture("input_video.mp4") back_sub = cv.createBackgroundSubtractorMOG2(history=500, varThreshold=16, detectShadows=True) while True: ret, frame = cap.read() if not ret: break fg_mask = back_sub.apply(frame) cv.imshow("Frame", frame) cv.imshow("Foreground Mask", fg_mask) if cv.waitKey(25) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() 2.8.3 Drawing Bounding Boxes Around Motion After creating a binary motion mask, contours can be detected. Very small contours are usually noise and should be ignored. Bounding boxes are drawn only around contours with area greater than a chosen threshold. import cv2 as cv cap = cv.VideoCapture(0) back_sub = cv.createBackgroundSubtractorMOG2() while True: ret, frame = cap.read() if not ret: break mask = back_sub.apply(frame) _, mask = cv.threshold(mask, 200, 255, cv.THRESH_BINARY) contours, _ = cv.findContours(mask, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE) for cnt in contours: if cv.contourArea(cnt) < 1000: continue x, y, w, h = cv.boundingRect(cnt) cv.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2) cv.putText(frame, "Motion", (x, y-10), cv.FONT_HERSHEY_SIMPLEX, 0.6, (0,255,0), 2) cv.imshow("Motion Detection", frame) if cv.waitKey(1) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() Table 2.9: Motion detection methods in OpenCV Method Advantages Limitations Frame differencing Very simple and fast; good for quick demonstrations. Sensitive to lighting and camera shake; may miss slow motion. Static background subtraction Good when background is stable. Needs clean background; affected by shadows. MOG2 background subtractor Adapts to gradual background changes; can detect shadows. Needs parameter tuning; shadow pixels may appear gray. KNN background subtractor Can handle complex scenes better in some cases. May be slower and needs tuning. Optical flow Estimates direction and magnitude of motion. More computationally expensive. Practical Tip: For accurate motion detection, keep the camera fixed, reduce sudden illumination changes, apply Gaussian blur to reduce noise, and use a minimum contour area threshold. 2.9 Object Tracking Using OpenCV Object tracking follows the same object across multiple frames after it has been selected or detected. A tracker usually starts with an initial bounding box and then estimates the updated object position in every new frame. Tracking is commonly used in surveillance, sports analytics, robotics, medical videos and human-computer interaction. Figure 2.4: Object tracking workflow from object selection to visual output 2.9.1 Motion Detection vs Object Detection vs Object Tracking Table 2.10: Difference between motion detection, object detection and object tracking Task Main Question Typical Output Example Motion Detection Where is movement? Motion mask or moving region. Detecting movement in a room. Object Detection What objects are present and where? Class labels and bounding boxes. Detecting persons, cars or faces. Object Tracking Where did the selected object move? Updated bounding box over time. Following one vehicle through a video. 2.9.2 Common Trackers OpenCV includes several classical tracking algorithms. The available functions may depend on the installed OpenCV version and whether the contrib package is installed. For teaching and practical work, CSRT, KCF and MIL are commonly discussed. Table 2.11: Common OpenCV object trackers Tracker Strength Limitation / Note MIL Generally available and simple to use. May drift when background is confusing. KCF Fast and suitable for many real-time cases. Less robust when object scale or appearance changes strongly. Tracker Strength Limitation / Note CSRT Often more accurate for single-object tracking. Slower than KCF. MOSSE Very fast and useful for high-FPS tracking. May be less accurate; often in legacy/contrib modules. MedianFlow Works well when object motion is smooth. Fails with fast motion or occlusion. GOTURN / Deep trackers Can use learned appearance models. May require model files and more resources. 2.9.3 Single Object Tracking Program The following program opens a video, lets the user select an object in the first frame, initializes a tracker, and draws the updated bounding box on every frame. import cv2 as cv cap = cv.VideoCapture("input_video.mp4") ret, frame = cap.read() if not ret: print("Cannot read video") exit() bbox = cv.selectROI("Select Object", frame, fromCenter=False, showCrosshair=True) cv.destroyWindow("Select Object") tracker = cv.TrackerCSRT_create() tracker.init(frame, bbox) while True: ret, frame = cap.read() if not ret: break success, bbox = tracker.update(frame) if success: x, y, w, h = [int(v) for v in bbox] cv.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2) cv.putText(frame, "Tracking", (x, y-10), cv.FONT_HERSHEY_SIMPLEX, 0.6, (0,255,0), 2) else: cv.putText(frame, "Tracking failure", (30, 50), cv.FONT_HERSHEY_SIMPLEX, 0.8, (0,0,255), 2) cv.imshow("Object Tracking", frame) if cv.waitKey(25) & 0xFF == ord('q'): break cap.release() cv.destroyAllWindows() 2.9.4 Choosing the Right Tracking Approach A tracker is useful when the object is already known and must be followed. However, if new objects enter the scene, object detection should be used first to locate them. Many practical systems combine detection and tracking: detection finds objects periodically, and tracking follows them between detections to reduce processing cost. Table 2.12: Choosing between detection and tracking Situation Recommended Approach Reason One selected object must be followed Single-object tracker Fast and simple after manual or automatic initialization. New objects appear at any time Object detector + tracker Detector identifies new objects; tracker maintains identity. Camera is fixed and only movement matters Motion detection No need to identify specific object class. High accuracy class labels are required Object detector Tracker does not classify objects by itself. Long video with occasional occlusion Detector-assisted tracking Detector can reinitialize after tracker failure. 2.9.5 Challenges in Object Tracking • Occlusion: the object becomes hidden behind another object. • Scale change: the object moves closer or farther from the camera. • Appearance change: lighting, rotation or deformation changes how the object looks. • Fast motion: the object moves too far between frames. • Background clutter: the object looks similar to the surrounding background. • Drift: the tracker slowly shifts away from the true object location. Key Insight: Tracking quality depends on the first bounding box. If the initial ROI includes too much background or misses part of the object, the tracker may fail quickly. 2.10 Further Reading / Viewing The following resources are recommended for extending the concepts covered in this chapter. Official Documentation