Initial commit
This commit is contained in:
St. Nebula
@@ -0,0 +1,109 @@
|
||||
---
|
||||
name: firebase-ai-logic
|
||||
description: Official skill for integrating Firebase AI Logic (Gemini API) into web applications. Covers setup, multimodal inference, structured output, and security.
|
||||
version: 1.0.0
|
||||
---
|
||||
|
||||
# Firebase AI Logic Basics
|
||||
|
||||
## Overview
|
||||
|
||||
Firebase AI Logic is a product of Firebase that allows developers to add gen AI to their mobile and web apps using client-side SDKs. You can call Gemini models directly from your app without managing a dedicated backend. Firebase AI Logic, which was previously known as "Vertex AI for Firebase", represents the evolution of Google's AI integration platform for mobile and web developers.
|
||||
|
||||
It supports the two Gemini API providers:
|
||||
- **Gemini Developer API**: It has a free tier ideal for prototyping, and pay-as-you-go for production
|
||||
- **Vertex AI Gemini API**: Ideal for scale with enterprise-grade production readiness, requires Blaze plan
|
||||
|
||||
Use the Gemini Developer API as a default, and only Vertex AI Gemini API if the application requires it.
|
||||
|
||||
## Setup & Initialization
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Before starting, ensure you have **Node.js 16+** and npm installed. Install them if they aren’t already available.
|
||||
- Identify the platform the user is interested in building on prior to starting: Android, iOS, Flutter or Web.
|
||||
- If their platform is unsupported, Direct the user to Firebase Docs to learn how to set up AI Logic for their application (share this link with the user https://firebase.google.com/docs/ai-logic/get-started)
|
||||
|
||||
### Installation
|
||||
|
||||
The library is part of the standard Firebase Web SDK.
|
||||
|
||||
`npm install -g firebase@latest`
|
||||
|
||||
If you're in a firebase directory (with a firebase.json) the currently selected project will be marked with "current" using this command:
|
||||
|
||||
`npx -y firebase-tools@latest projects:list`
|
||||
|
||||
Ensure there's at least one app associated with the current project
|
||||
|
||||
`npx -y firebase-tools@latest apps:list`
|
||||
|
||||
Initialize AI logic SDK with the init command
|
||||
|
||||
`npx -y firebase-tools@latest init # Choose AI logic`
|
||||
|
||||
This will automatically enable the Gemini Developer API in the Firebase console.
|
||||
|
||||
More info in [Firebase AI Logic Getting Started](https://firebase.google.com/docs/ai-logic/get-started.md.txt)
|
||||
|
||||
## Core Capabilities
|
||||
|
||||
### Text-Only Generation
|
||||
|
||||
### Multimodal (Text + Images/Audio/Video/PDF input)
|
||||
|
||||
Firebase AI Logic allows Gemini models to analyze image files directly from your app. This enables features like creating captions, answering questions about images, detecting objects, and categorizing images. Beyond images, Gemini can analyze other media types like audio, video, and PDFs by passing them as inline data with their MIME type. For files larger than 20 megabytes (which can cause HTTP 413 errors as inline data), store them in Cloud Storage for Firebase and pass their URLs to the Gemini Developer API.
|
||||
|
||||
### Chat Session (Multi-turn)
|
||||
|
||||
Maintain history automatically using `startChat`.
|
||||
|
||||
### Streaming Responses
|
||||
|
||||
To improve the user experience by showing partial results as they arrive (like a typing effect), use `generateContentStream` instead of `generateContent` for faster display of results.
|
||||
|
||||
### Generate Images with Nano Banana
|
||||
|
||||
- Start with Gemini for most use cases, and choose Imagen for specialized tasks where image quality and specific styles are critical. (Example: gemini-2.5-flash-image)
|
||||
- Requires an upgraded Blaze pay-as-you-go billing plan.
|
||||
|
||||
### Search Grounding with the built in googleSearch tool
|
||||
|
||||
## Supported Platforms and Frameworks
|
||||
|
||||
Supported Platforms and Frameworks include Kotlin and Java for Android, Swift for iOS, JavaScript for web apps, Dart for Flutter, and C Sharp for Unity.
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Structured Output (JSON)
|
||||
|
||||
Enforce a specific JSON schema for the response.
|
||||
|
||||
### On-Device AI (Hybrid)
|
||||
|
||||
Hybrid on-device inference for web apps, where the Firebase Javascript SDK automatically checks for Gemini Nano's availability (after installation) and switches between on-device or cloud-hosted prompt execution. This requires specific steps to enable model usage in the Chrome browser, more info in the [hybrid-on-device-inference documentation](https://firebase.google.com/docs/ai-logic/hybrid-on-device-inference.md.txt).
|
||||
|
||||
## Security & Production
|
||||
|
||||
### App Check
|
||||
|
||||
Recommended: The developer must enable Firebase App Check to prevent unauthorized clients from using their API quota. see [App-check recaptcha enterprise](https://firebase.google.com/docs/app-check/web/recaptcha-enterprise-provider.md.txt).
|
||||
|
||||
### Remote Config
|
||||
|
||||
Consider that you do not need to hardcode model names (e.g., `gemini-flash-lite-latest`). Use Firebase Remote Config to update model versions dynamically without deploying new client code. See [Changing model names remotely](https://firebase.google.com/docs/ai-logic/change-model-name-remotely.md.txt)
|
||||
|
||||
## Initialization Code References
|
||||
|
||||
| Language, Framework, Platform | Gemini API provider | Context URL |
|
||||
| :---- | :---- | :---- |
|
||||
| Web Modular API | Gemini Developer API (Developer API) | firebase://docs/ai-logic/get-started |
|
||||
|
||||
**Always use the most recent version of Gemini (gemini-flash-latest) unless another model is requested by the docs or the user. DO NOT USE gemini-1.5-flash**
|
||||
|
||||
## References
|
||||
|
||||
[Web SDK code examples and usage patterns](references/usage_patterns_web.md)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
# Firebase AI Logic Basics
|
||||
|
||||
## Initialization Pattern
|
||||
You must initialize the ai-logic service after the main Firebase App.
|
||||
```JavaScript
|
||||
import { initializeApp } from "firebase/app";
|
||||
import { getAI, getGenerativeModel, GoogleAIBackend } from "firebase/ai";
|
||||
|
||||
|
||||
// If running in Firebase App Hosting, you can skip Firebase Config and instead use:
|
||||
// const app = initializeApp();
|
||||
|
||||
const firebaseConfig = {
|
||||
// ... your firebase config
|
||||
};
|
||||
|
||||
const app = initializeApp(firebaseConfig);
|
||||
|
||||
// Initialize the AI Logic service (defaults to Gemini Developer API)
|
||||
// To set the AI provider, set the backend as the second parameter
|
||||
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
|
||||
|
||||
const generationConfig = {
|
||||
candidate_count: 1,
|
||||
maxOutputTokens: 2048,
|
||||
stopSequences: [],
|
||||
temperature: 0.7, // Balanced: creative but focused
|
||||
topP: 0.95, // Standard: allows a wide range of probable tokens
|
||||
topK: 40, // Standard: considers the top 40 tokens
|
||||
};
|
||||
|
||||
// Specify the config as part of creating the `GenerativeModel` instance
|
||||
const model = getGenerativeModel(ai, { model: "gemini-2.5-flash-lite", generationConfig });
|
||||
```
|
||||
|
||||
## Core Capabilities
|
||||
Text-Only Generation
|
||||
```JavaScript
|
||||
async function generateText(prompt) {
|
||||
const result = await model.generateContent(prompt);
|
||||
const response = await result.response;
|
||||
return response.text();
|
||||
}
|
||||
```
|
||||
|
||||
## Multimodal (Text + Images/Audio/Video/PDF input)
|
||||
Firebase AI Logic accepts Base64 encoded data or specific file references.
|
||||
```JavaScript
|
||||
// Helper to convert file to base64 generic object
|
||||
async function fileToGenerativePart(file) {
|
||||
const base64EncodedDataPromise = new Promise((resolve) => {
|
||||
const reader = new FileReader();
|
||||
reader.onloadend = () => resolve(reader.result.split(',')[1]);
|
||||
reader.readAsDataURL(file);
|
||||
});
|
||||
|
||||
return {
|
||||
inlineData: {
|
||||
data: await base64EncodedDataPromise,
|
||||
mimeType: file.type,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function analyzeImage(prompt, imageFile) {
|
||||
const imagePart = await fileToGenerativePart(imageFile);
|
||||
const result = await model.generateContent([prompt, imagePart]);
|
||||
return result.response.text();
|
||||
}
|
||||
```
|
||||
|
||||
## Chat Session (Multi-turn)
|
||||
Maintain history automatically using startChat.
|
||||
```JavaScript
|
||||
const chat = model.startChat({
|
||||
history: [
|
||||
{
|
||||
role: "user",
|
||||
parts: [{ text: "Hello, I am a developer." }],
|
||||
},
|
||||
{
|
||||
role: "model",
|
||||
parts: [{ text: "Great to meet you. How can I help with code?" }],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
async function sendMessage(msg) {
|
||||
const result = await chat.sendMessage(msg);
|
||||
return result.response.text();
|
||||
}
|
||||
```
|
||||
|
||||
## Streaming Responses
|
||||
For real-time UI updates (like a typing effect).
|
||||
```JavaScript
|
||||
async function streamResponse(prompt) {
|
||||
const result = await model.generateContentStream(prompt);
|
||||
for await (const chunk of result.stream) {
|
||||
const chunkText = chunk.text();
|
||||
console.log("Stream chunk:", chunkText);
|
||||
// Update UI here
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Generate Images with Nano Banana
|
||||
|
||||
```Javascript
|
||||
import { initializeApp } from "firebase/app";
|
||||
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";
|
||||
|
||||
|
||||
// Initialize FirebaseApp
|
||||
const firebaseApp = initializeApp(firebaseConfig);
|
||||
|
||||
// Initialize the Gemini Developer API backend service
|
||||
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });
|
||||
|
||||
// Create a `GenerativeModel` instance with a model that supports your use case
|
||||
const model = getGenerativeModel(ai, {
|
||||
model: "gemini-2.5-flash-image",
|
||||
// Configure the model to respond with text and images (required)
|
||||
generationConfig: {
|
||||
responseModalities: [ResponseModality.TEXT, ResponseModality.IMAGE],
|
||||
},
|
||||
});
|
||||
|
||||
// Provide a text prompt instructing the model to generate an image
|
||||
const prompt = 'Generate an image of the Eiffel Tower with fireworks in the background.';
|
||||
|
||||
// To generate an image, call `generateContent` with the text input
|
||||
const result = model.generateContent(prompt);
|
||||
|
||||
// Handle the generated image
|
||||
try {
|
||||
const inlineDataParts = result.response.inlineDataParts();
|
||||
if (inlineDataParts?.[0]) {
|
||||
const image = inlineDataParts[0].inlineData;
|
||||
console.log(image.mimeType, image.data);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('Prompt or candidate was blocked:', err);
|
||||
}
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
Structured Output (JSON)
|
||||
Enforce a specific JSON schema for the response.
|
||||
```JavaScript
|
||||
import { getGenerativeModel, Schema } from "firebase/ai";
|
||||
const jsonModel = getGenerativeModel(ai, {
|
||||
model: "gemini-2.5-flash-lite",
|
||||
generationConfig: {
|
||||
responseMimeType: "application/json",
|
||||
// Optional: Define a schema
|
||||
schema = Schema.object({ ... });
|
||||
}
|
||||
});
|
||||
|
||||
async function getJsonData(prompt) {
|
||||
const result = await jsonModel.generateContent(prompt);
|
||||
return JSON.parse(result.response.text());
|
||||
}
|
||||
```
|
||||
|
||||
On-Device AI (Hybrid)
|
||||
Automatically switch between local Gemini Nano and cloud models based on device capability.
|
||||
```JavaScript
|
||||
import {getGenerativeModel, InferenceMode } from "firebase/ai";
|
||||
|
||||
const hybridModel = getGenerativeModel(ai, { mode: InferenceMode.PREFER_ON_DEVICE });
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user