我在使用@google/generative-ai时,正在寻找一种从URL上传视频的解决方案。我已经弄清楚了如何使用存储在我机器上的视频,但对于外部来源的视频还不清楚该如何操作。
这是我当前用于向@google/generative-ai上传视频的函数:
"use server"const { GoogleGenerativeAI } = require("@google/generative-ai");import { GoogleAIFileManager, FileState } from "@google/generative-ai/server";import { redirect } from "next/navigation";import fetchVideoById from "./fetchVideoById";// Initialize GoogleAIFileManager with your API_KEY.const fileManager = new GoogleAIFileManager(process.env.API_KEY);// Access your API key as an environment variable (see "Set up your API key" above)const genAI = new GoogleGenerativeAI(process.env.API_KEY);// Choose a Gemini model.const model = genAI.getGenerativeModel({ model: "gemini-1.5-pro",});export async function generateSummary(formData) { const rows = await fetchVideoById(formData.get("id")) const url = rows["url"] // by the way, this returns the URL for the video that is stored in my database, so assume that the URL is valid. console.log("Uploading file...") const fileManager = new GoogleAIFileManager(process.env.API_KEY); // Upload the file and specify a display name. const uploadResponse = await fileManager.uploadFile(url, { mimeType: "video/mp4", displayName: rows["title"], }); // View the response. console.log(`Uploaded file ${uploadResponse.file.displayName} as: ${uploadResponse.file.uri}`); const name = uploadResponse.file.name; // Poll getFile() on a set interval (10 seconds here) to check file state. let file = await fileManager.getFile(name); while (file.state === FileState.PROCESSING) { process.stdout.write(".") // Fetch the file from the API again file = await fileManager.getFile(name) } if (file.state === FileState.FAILED) { throw new Error("Video processing failed."); } // When file.state is ACTIVE, the file is ready to be used for inference. console.log(`File ${file.displayName} is ready for inference as ${file.uri}`); const result = await model.generateContent([ { fileData: { mimeType: uploadResponse.file.mimeType, fileUri: uploadResponse.file.uri } }, { text: "Summarize this video." }, ]); // Handle the response of generated text console.log(result.response.text()) return result.response.text() console.log("Deleting file...") await fileManager.deleteFile(file.name); console.log("Deleted file.") }
这是我得到的错误信息:
Error: ENOENT: no such file or directory, open 'C:\Users\n_mac\Desktop\Coding\summa\front-end\https:\m9x5emw6q3oaze3r.public.blob.vercel-storage.com\monkeyman64\6999DBC5-2D93-4220-BC43-3C16C9A5D9C6-IZzFC1THZXPSgeAK1NPo3uCVxA091l.mp4'
如您所见,它在我的机器上搜索文件,而文件实际上存储在Vercel Blob中。
任何帮助都将不胜感激。
回答:
当我查看uploadFile
脚本时,似乎文件是以multipart
形式上传的。参考 而且,似乎filePath
需要是本地PC的文件路径。不幸的是,在当前阶段,似乎无法直接使用URL。
那么,在这种情况下,如何从URL下载数据并上传到Gemini而不创建文件呢?当这种方法反映在脚本中时,以下修改如何?
从:
const fileManager = new GoogleAIFileManager(process.env.API_KEY);// Upload the file and specify a display name.const uploadResponse = await fileManager.uploadFile(url, { mimeType: "video/mp4", displayName: rows["title"],});// View the response.console.log(`Uploaded file ${uploadResponse.file.displayName} as: ${uploadResponse.file.uri}`);const name = uploadResponse.file.name;
到:
const fileManager = new GoogleAIFileManager(process.env.API_KEY);// Download data.const res = await fetch(url);const buffer = await res.arrayBuffer();// Upload the downloaded data.const formData = new FormData();const metadata = { file: { mimeType: "video/mp4", displayName: rows["title"] } };formData.append("metadata", new Blob([JSON.stringify(metadata)], { contentType: "application/json" }));formData.append("file", new Blob([buffer], { type: "video/mp4" }));const res2 = await fetch( `https://generativelanguage.googleapis.com/upload/v1beta/files?uploadType=multipart&key=${fileManager.apiKey}`, { method: "post", body: formData });const uploadResponse = await res2.json();// View the response.console.log(`Uploaded file ${uploadResponse.file.displayName} as: ${uploadResponse.file.uri}`);const name = uploadResponse.file.name;
- 在这一修改中,使用了Node.js的fetch API。
- 当运行此脚本时,我确认了可以从URL正确地将MP4数据上传到Gemini。
注意:
- 在这一修改中,假设您的URL可以直接下载MP4数据。请注意这一点。