Transforming the Driver Experience with MongoDB & Google Cloud

On-Board Components

These run in the vehicle, close to the driver, and enable real-time voice interaction.

• Car Console: The in-car interface where users speak to the assistant and get responses. It’s a web app in this demo, but represents the embedded system in a real vehicle.

• Local Data Storage: Vehicles store key signals locally using PowerSync SDK, a lightweight edge database built on SQLite. This ensures fast access to diagnostic data and keeps data synced with MongoDB Atlas.

• Assistant Backend: This component manages the conversation. It handles voice transcription using Google Cloud Speech-to-Text and reads the driver’s intent. Depending on the query, it may respond directly or call tools to fetch more data or take action. For the demo version, we included four sample tools:

  • Consult Manual: Uses Atlas Vector Search to retrieve relevant info from the car manual.

  • Run Diagnostics: Fetches current diagnostic codes from local vehicle data.

  • Recalculate Route: Adjusts the trip if the driver adds a stop.

  • Close Chat: Ends the conversation gracefully.

const functionDeclarations = [
  {
    functionDeclarations: [
      {
        name: "closeChat",
        description:
          "Closes the chat window when the conversation is finished. By default it always returns to the navigation view. Ask the user to confirm this action before executing.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            view: {
              type: FunctionDeclarationSchemaType.STRING,
              enum: ["navigation"],
              description:
                "The next view to display after closing the chat.",
            },
          },
          required: ["view"],
        },
      },
      {
        name: "recalculateRoute",
        description:
          "Recalculates the route when a new stop is added. By default this function will find the nearest service station. Ask the user to confirm this action before executing.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {},
        },
      },
      {
        name: "consultManual",
        description:
          "Retrieves relevant information from the car manual.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            query: {
              type: FunctionDeclarationSchemaType.STRING,
              description:
                "A question that represents an enriched version of what the user wants to retrieve from the manual. It must be in the form of a question.",
            },
          },
          required: ["query"],
        },
      },
      {
        name: "runDiagnostic",
        description:
          "Fetches active Diagnostic Trouble Codes (DTCs) in the format OBD II (SAE-J2012DA_201812) from the vehicle to assist with troubleshooting.",
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {},
        },
      },
    ],
  },
];

Cloud Components

These provide AI intelligence, scalable storage, and data processing capabilities.

• Data Ingestion: Unstructured content like car manuals is uploaded to Google Cloud Storage. This triggers a pipeline using Pub/Sub, Cloud Run, and Document AI to split the PDFs into chunks. Vertex AI generates embeddings for these chunks, which are then stored in MongoDB Atlas for semantic search.

• Serving Speech APIs: Google Cloud’s Text-to-Speech and Speech-to-Text handle natural voice interaction. Vertex AI: Provides text embeddings for search queries and powers the LLM (Gemini) used by the assistant.

• Data Storage and Retrieval: MongoDB Atlas stores:

  • Manual chunk embeddings for retrieval via Atlas Vector Search.

  • User preferences and session data.

  • Vehicle signals—both latest values and full time series telemetry.

Atlas Vector Search is used to match user questions with the most relevant manual sections, enabling a Retrieval-Augmented Generation (RAG) flow. MongoDB’s native support for structured, semi-structured, and vector data in one place simplifies the assistant logic and speeds up development.

• Data Sync: We use PowerSync for two-way sync between the vehicle and cloud:

  • Vehicle to Cloud: Vehicle sends telemetry data like diagnostic codes, speed, or acceleration. A Cloud Run function processes and stores this in Atlas.

  • Cloud to Vehicle: Enables updates or actions sent remotely to the car—like OTA updates or remote locking.

The Role of MongoDB in Conversational AI

MongoDB Atlas is the developer data platform that powers this solution. Here's how it improves the architecture:

• Unifies operational and vector data: Vehicle signals, vector embeddings, and user sessions are stored together in a single, consistent platform.

• Enables more relevant responses: Atlas Vector Search retrieves the right chunks from large documents instantly, fueling accurate and context-rich responses.

• Built for the enterprise scale: Whether it's one model or a global fleet, MongoDB Atlas offers built-in horizontal scalability, high-availability and enterprise-grade security.

• Simplifies edge and cloud sync: PowerSync and MongoDB work together to bridge in-car and cloud environments without friction.

This architecture is designed to scale, evolve, and adapt—just like the vehicles it supports. With MongoDB at the core, automakers can focus less on data plumbing and more on delivering smart, helpful in-car experiences that truly make a difference on the road.

Built for Innovation and Speed

The document model is designed for how developers think. Need to add metadata to your diagnostic data? Or associate multiple content types with a single manual section? In MongoDB, that’s just a few lines of code—not a schema rewrite. And because each document is self-contained, queries are faster and simpler. Fewer joins, fewer moving parts, and more time to build.

This flexibility also unlocks faster innovation cycles. As new vehicle features roll out or user expectations shift, teams can evolve the data model on the fly without expensive migrations or app downtime.

The Natural Choice for AI Workloads

Generative AI thrives on rich, diverse, and unstructured data. Embeddings, contextual metadata, structured references—all of it plays a part in making AI systems smarter. MongoDB is uniquely suited for these kinds of applications:

• Store vector embeddings, metadata, and source content in a single document.

• Combine structured and vector data without jumping between systems.

• Query vector and non-vector fields together for contextual, accurate results.

The result? Simpler architecture, better performance, and more relevant AI responses.

Example 1: Car Manual Embeddings

When using a retrieval-augmented generation (RAG) approach, the quality of the chunking and embeddings directly impacts the quality of the AI’s responses. Poorly segmented content or missing context can lead to vague or inaccurate answers. Technical manuals are inherently complex. They often contain dense text, diagrams, and domain-specific terminology—making it challenging to retrieve the right information.

To address this, we represent each chunk of the manual as a document. The document includes not just the text and its vector embedding, but also metadata such as content type (e.g., safety, diagnostics), page numbers, chunk length, and links to related chunks. This additional context helps the system understand how pieces of information relate to one another—especially important in highly technical or interdependent topics.

MongoDB’s flexible document model makes it straightforward to capture this complexity. As the manual evolves or as new needs emerge, we can incrementally add fields or adjust structure without requiring a full schema migration. This enables more precise retrieval and ultimately more helpful AI responses.

{
  "_id": {
    "$oid": "67cc4b09c128338a8133b59a"
  },
  "text": "Oil Pressure Warning Lamp. If it illuminates when the engine is running this indicates a malfunction. Stop your vehicle as soon as it is safe to do so and switch the engine off. Check the engine oil level. If the oil level is sufficient, this indicates a system malfunction.",
  "page_numbers": [
    23
  ],
  "content_type": [
    "safety",
    "diagnostic"
  ],
  "metadata": {
    "page_count": 1,
    "chunk_length": 1045
  },
  "id": "chunk_0053",
  "prev_chunk_id": "chunk_0052",
  "next_chunk_id": "chunk_0054",
  "related_chunks": [
    {
      "id": "chunk_0048",
      "content_type": [
        "safety"
      ],
      "relation_type": "same_context"
    },
    {
      "id": "chunk_0049",
      "content_type": [
        "safety"
      ],
      "relation_type": "same_context"
    },
    ...
  ],
  "embedding": [
    -0.002636542310938239,
    -0.005587903782725334,
    ...
  ],
  "embedding_timestamp": "2025-03-08T13:50:00.887107"
}

Example 2: Vehicle Signal Data

For vehicle signals, we modeled our data using the COVESA Vehicle Signal Specification (VSS). VSS provides a standardized, hierarchical structure to describe real-time signals—like speed, acceleration, or diagnostic trouble codes (DTCs). It’s an open, extensible format that enables easier collaboration, system integration, and data reuse across vehicle platforms.

MongoDB document model natively handles nested structures, representing the VSS hierarchy is straightforward. Signals can be grouped logically, just like they appear in the VSS model, which aligns perfectly with the tree-based structure of the spec.

Figure 3. The VSS data model is a hierarchical tree structure built with modules that can be flexibly combined. Source: https://covesa.global/vehicle-signal-specification/

This structure not only accelerates development but also ensures that AI tools and workflows have consistent access to clean, structured, and meaningful data.

{
  "_id": {
    "$oid": "67e58d5f672b23090e57d478"
  },
  "VehicleIdentification": {
    "VIN": "1HGCM82633A004352"
  },
  "Speed": 0,
  "TraveledDistance": 0,
  "CurrentLocation": {
    "Timestamp": "2020-01-01T00:00:00Z",
    "Latitude": 0,
    "Longitude": 0,
    "Altitude": 0
  },
  "Acceleration": {
    "Lateral": 0,
    "Longitudinal": 0,
    "Vertical": 0
  },
  "Diagnostics": {
    "DTCCount": 0,
    "DTCList": []
  }
}

MongoDB’s document model doesn’t just store your data. It mirrors the complexity of the real world—making it easier to build smarter systems that respond in real time, adapt to user needs, and grow with your platform. Whether you’re storing vehicle diagnostics or vector-encoded manuals, MongoDB gives you the tools to build intelligent experiences faster.

MongoDB Developer Data Platform

• MongoDB Atlas Database

• MongoDB Atlas Vector Search

• MongoDB Node Driver

• Aggregation Pipeline

• Change Streams

Partner Technologies

• Google Cloud

• PowerSync

• Vercel