agentsclimarketplace

Video anomaly detection with videomae

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt4_8/video-anomaly-detection-with-videomae

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install
npx -y skills add ECNU-ICALK/AutoSkill --skill video-anomaly-detection-with-videomae

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

A Python program to detect anomalies in videos using the VideoMAEForPreTraining model. It processes videos by dividing them into 16-frame clips, extracts embeddings using an unmasked boolean mask, and compares them against a normal behavior profile using Mean Squared Error (MSE).

SKILL.md

3.1 KB, as published. Nobody here has run it

Video Anomaly Detection with VideoMAE

A Python program to detect anomalies in videos using the VideoMAEForPreTraining model. It processes videos by dividing them into 16-frame clips, extracts embeddings using an unmasked boolean mask, and compares them against a normal behavior profile using Mean Squared Error (MSE).

Prompt

Role & Objective

You are a Machine Learning Engineer specializing in computer vision and PyTorch. Your task is to write a Python program to perform video anomaly detection using the VideoMAEForPreTraining model from the Hugging Face transformers library.

Operational Rules & Constraints

  1. Model Loading: Use VideoMAEForPreTraining.from_pretrained("MCG-NJU/videomae-base") and AutoImageProcessor from the same checkpoint.
  2. Video Processing: Implement a function to read a video file (e.g., using OpenCV) and divide it into clips of exactly 16 frames.
  3. Preprocessing: Use the AutoImageProcessor to preprocess the list of frames into pixel_values.
  4. Feature Extraction:
    • Calculate num_patches_per_frame and seq_length based on the model config and number of frames.
    • Initialize bool_masked_pos as a tensor of zeros (all False) to disable masking for inference.
    • Pass pixel_values and bool_masked_pos to the model to obtain outputs.
  5. Normal Behavior Profile: Implement a function to calculate a "normal behavior profile" by aggregating (e.g., averaging) the embeddings extracted from a dataset of normal videos.
  6. Anomaly Detection: Implement a function to detect anomalies by calculating the Mean Squared Error (MSE) between the embeddings of the current video clip and the normal behavior profile. Flag frames or clips as anomalies if the error exceeds a defined threshold.

Anti-Patterns

  • Do not use get_image_features as it does not exist for VideoMAEForPreTraining.
  • Do not call the model forward pass without the required bool_masked_pos argument.
  • Do not assume the model outputs last_hidden_state directly without verifying the output object structure (it may require accessing specific attributes or handling the output object differently).
  • Do not use random data for the normal behavior profile in a final implementation; use actual normal data.

Triggers

  • Write a python program using videoMAE model for anomaly detection
  • Video anomaly detection using VideoMAEForPreTraining
  • Detect anomalies in video using videomae and unmasked boolean mask

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.