AI模型之文字生图/文字生视频
前言由于公司的业务都是关于AI的很多都是文字或者图生图或者是文字生视频等等开始1.huggingface网站2.查看显卡显存配置ps所有的操作都是要基于你的电脑配置进行的# 查看你的显卡和显存 nvidia-smi如果使用huggingface的话要经历本地环境设置和线上模型授权1.本地环境设置下载的模型默认会装在c盘所以很容易会导致c盘爆满ps如果已经下载了可以进行迁移就是把C盘的C:\Users\你的用户名\.cache\huggingface复制到你需要的盘里面然后设置环境变量即可HF_HOMED:\huggingface HF_HUB_CACHED:\huggingface\hub HF_DATASETS_CACHED:\huggingface\datasets2.线上授权登录你huggingface账号没有就注册一个点击agree and access repository进行授权即可文字生图模型1.安装依赖# 安装对应合适的torch pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128 # 安装生成图片需要的依赖库 pip install -U diffusers transformers accelerate bitsandbytes sentencepiece protobuf huggingface_hub # 由于是使用低版本的 black-forest-labs/FLUX.1-schnell 所以需要指定版本 bitsandbytes0.46.1 pip install -U bitsandbytes0.46.12.由于第一次使用huggingfacehf auth login # 提示授权信息的目录在哪里【如果你没有配置的话 就是在c盘模型下载也是在c盘】 The token oauth-liangxiaotian has been saved to C:\Users\16330\.cache\huggingface\stored_tokens Your token has been saved to C:\Users\16330\.cache\huggingface\token Login successful. The current active token is: oauth-liangxiaotian3.编写main.pyimport time import torch from diffusers import DiffusionPipeline from diffusers.quantizers import PipelineQuantizationConfig MODEL_ID black-forest-labs/FLUX.1-schnell # Transformer T5 text encoder 做 4bit NF4 quant_config PipelineQuantizationConfig( quant_backendbitsandbytes_4bit, quant_kwargs{ load_in_4bit: True, bnb_4bit_quant_type: nf4, bnb_4bit_compute_dtype: torch.bfloat16, bnb_4bit_use_double_quant: True, }, components_to_quantize[ transformer, text_encoder_2, ], ) print(Loading FLUX.1-schnell 4bit) pipe DiffusionPipeline.from_pretrained( MODEL_ID, torch_dtypetorch.bfloat16, quantization_configquant_config, ) # 低显存机器推荐 pipe.enable_model_cpu_offload() prompt A cinematic photograph of a futuristic Chinese city at night, neon signs reflected on wet streets, a small red sports car, highly detailed, realistic lighting, 35mm photography generator torch.Generator(cpu).manual_seed(42) torch.cuda.reset_peak_memory_stats() start time.time() image pipe( promptprompt, width768, height768, num_inference_steps4, guidance_scale0.0, max_sequence_length256, generatorgenerator, ).images[0] elapsed time.time() - start image.save(flux_schnell_4bit_test_1.png) print(Generation finished) print(fTime: {elapsed:.2f} sec) if torch.cuda.is_available(): peak torch.cuda.max_memory_allocated() / 1024**3 print(fPeak allocated VRAM: {peak:.2f} GB) print(Saved: flux_schnell_4bit_test_1.png)文字生视频模型注意点要查看自己的显卡是否支持我是3060显卡6g显存所以~选择使用Wan2.1的Wan2.1-T2V-1.3B模型下载项目下载模型使用模型1.下载项目git clone https://github.com/Wan-Video/Wan2.1.git cd Wan2.1 安装依赖 pip install -r requirements.txt 还要安装 huggingface_hub pip install huggingface_hub[cli] hf -v2.下载模型下载模型下载到本地 hf download Wan-AI/Wan2.1-T2V-1.3B --local-dir .\Wan2.1-T2V-1.3B3.使用模型sh python generate.py --task t2v-1.3B --size 832*480 --ckpt_dir .\Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --prompt Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage. 更节约电脑性能 sh python generate.py --task t2v-1.3B --size 480*832 --ckpt_dir .\Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --sample_steps 10 --frame_num 17 --prompt Two cats boxing on a stage.
