CogVideoX是智谱AI开源的一款视频生成模型，其核心在于3D变分自编码器和3D旋转位置编码（3D RoPE）技术。这些技术使得CogVideoX能够在保持视频帧间连贯性的同时，极大地降低计算资源需求，并生成流畅、连贯的视频序列。此外，CogVideoX还具备强大的文本理解能力，能够处理超长且复杂的文本提示，生成与用户输入高度相关的视频内容.

查看显存

nvidia-smi

在线创建Python虚拟环境

 python3.10 -m venv python310

激活Python虚拟环境

 #source /usr/local/bin/python310/bin/activate

拉取CogVideo代码

#cd /home

git clone https://github.com/THUDM/CogVideo.git

安装CogVideo的依赖

pip install -r requirements.txt  

modelscope下载cogvideox-2b模型

pip install modelscope

modelscope download --model zhipuai/cogvideox-2b

模型具体位置：

/home/admin/workspace/.cache/modelscope/hub/zhipuai/cogvideox-2b

创建test.py

vi test.py

import torch

from diffusers import CogVideoXPipeline

from diffusers.utils import export_to_video

import os

os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'



# prompt里写自定义想要生成的视频内容

prompt = "A panda, dressed in a small, red jacket and a tiny hat, sits on a wooden stool in a serene bamboo forest. The panda's fluffy paws strum a miniature acoustic guitar, producing soft, melodic tunes. Nearby, a few other pandas gather, watching curiously and some clapping in rhythm. Sunlight filters through the tall bamboo, casting a gentle glow on the scene. The panda's face is expressive, showing concentration and joy as it plays. The background includes a small, flowing stream and vibrant green foliage, enhancing the peaceful and magical atmosphere of this unique musical performance."



pipe = CogVideoXPipeline.from_pretrained(

    "/home/admin/workspace/.cache/modelscope/hub/zhipuai/cogvideox-2b", # 这里填CogVideo模型存放的位置，此处是放在了数据盘中

    torch_dtype=torch.float16

).to("cuda")



# 参数do_classifier_free_guidance设置为True可以启用无分类器指导，增强生成内容一致性和多样性

# num_videos_per_prompt控制每个prompt想要生成的视频数量

# max_sequence_length控制输入序列的最大长度

prompt_embeds, _ = pipe.encode_prompt(

    prompt=prompt,

    do_classifier_free_guidance=True,

    num_videos_per_prompt=1,

    max_sequence_length=226,

    device="cuda",

    dtype=torch.float16,

)

# 释放不需要的张量

del _

torch.cuda.empty_cache()



# 生成视频

video = pipe(

    num_inference_steps=10,  # 减少推理步骤

    guidance_scale=4,  # 降低指导比例

    prompt_embeds=prompt_embeds,

).frames[0]

#使用较少的推理步骤和较低的指导比例来减少显存使用。

export_to_video(video, "output.mp4", fps=8)

运行test.py

#python test.py

模型对比