Boost Prisma query performance: Add non-unique foreign key indexes, verify with 'explain query' for speed!

Optimize DB queries with user-specific note indexes, boosting performance and reducing scan times.

Optimizing Database Queries with Indexes

[00:00] So I'm going to use the CLI, we'll say SQLite 3, and point it to Prisma DataDB. And then I'll paste in our explain query. And here's that. Explain query plan, select star from note where owner ID equals 1. And our query plan is to scan the note. So a scan's not always a terrible thing.

[00:19] Like it's not instantly, oh, it's a scan, it must be bad. But in our particular case, we're going to be getting lots and lots of notes. And so if any time we wanted to find a note for a single user, you can think of scan as like we're looking at every single record, right? So if we're having to look at every single record,

[00:36] and when our plans for notes is to have many of them, then that seems inefficient, right? So it would probably be a good idea to add a index to that. So let's go to our Prisma schema here. And we're going to add a couple of indexes to our models here.

[00:55] So we'll add an index for the owner ID right here. And the reason we have to do that is because the owner ID is not unique. If we take a look at our migration file here, then we'll see we actually do already have a couple of indexes that Prisma creates for us. And that's for all of the unique fields that we have.

[01:14] So the user has a unique on the email and on the username. And then the user image, it's a one-to-one relationship. So we have a at unique on the user ID. But for this one-to-many, where one user can have many notes, we're not going to have at unique on this. That wouldn't make sense.

[01:33] So we're going to just tell our database, hey, we need to have an index. Make a special record or special place where you have an index for all of the notes by their owner ID. So we'll add index, and that's going to be owner ID.

[01:50] I don't care to specify a name, so thanks anyway, Copilot. So we'll save that. And then let's add one for all of the other models. I just do this as like a default. It's pretty much a pretty solid default for most of the databases to add an index to all foreign keys that are not unique.

[02:08] So index here on the note ID. And also always a good idea when you're doing any sort of performance thing at all to add a comment. So we're going to say non-unique foreign key. And we'll do the same thing on this one as well.

[02:29] And then we don't need an index here because again, the user ID is unique. We have that index right there anyway. All right, so with that now I can say, let's pull up another terminal, npx prisma migrate dev. And we're going to make a new migration, foreign keys.

[02:48] And let's take a look at the migration that was created here. So it created an index for the note owner ID and the note image note ID there as well. And so with that now, if I rerun that explain query plan, and there's our select, then we're going to still get a scan note.

[03:07] And that might be because I need to actually perform the migration. Let's see, yeah, our database is already in sync, but it could be that I need to turn this off and start it up again. So let's get our explain query plan, stick that in there and copy that, stick that in there.

[03:29] And now we're getting what we wanted. We can search the note using index note owner ID, IDX. If you're curious what indexes you have in a given database, you can actually select that from a special table. So we're going to say select name from the SQLite underscore master,

[03:49] where the type is index. And that's going to give us all of the indexes that we've got. We've got auto indexes that are created for us. And then we have the index on those unique fields. And then those indexes we just created ourselves. So at any time, if you want to look at what indexes are available,

[04:08] then that is how you accomplish that. So our query is definitely going to be faster. Normally this is something that you want to try and measure somehow. So maybe create a bajillion notes or something and then test that out and like look at querying a specific user's notes. So like we could go to Cody's notes

[04:27] and measure how long that query is going to take. And then of course you also have logging going on as well. So then you can take a look at those logs as you're navigating to these different pages and just make sure that the queries are shorter than they were before you made the change. So it's a good idea to make sure you measure

[04:44] before you actually implement these indexes. But at the same time, an index and a non-unique foreign key is almost always a good bet. And so I pretty much could say like 99% of the time you should do that. So that is adding a unique or a index

[05:03] on a foreign key that is not unique.

Optimizing Query Performance with Indexes in Prisma

[00:00] For this exercise, we're going to be optimizing a query of the notes by a owner ID. And you're gonna be adding an index for foreign keys to all of the models that have foreign keys, where those foreign keys are not unique. If they're unique, they already have an index that was created for us by Prisma.

[00:19] But if they're not unique, then we need to add an index. So your job is to first understand what's going on. And so you're gonna be using explain query, whether that's using SQLite 3 or the script file that I showed you how to do. And once you understand what's going on there and you're seeing the scan in there and all that stuff,

[00:38] then you're gonna add the index and do the explain again and make sure that scan does not show up and that we can use that index. And this we're gonna do so we can keep our queries really fast. So give that a whirl and we'll see you when you're done.

Data modeling is as a critical skill for the full stack developer. After all, data is the foundation of your application.

If you don't do it right, you'll find yourself stuck before you even get started. A poorly designed database schema can lead to a lot of headaches down the road when you need to make changes or add new features. Inefficient queries can slow down your application, leading to a poor user experience. Analyzing root causes to issues like these can be a time-consuming and frustrating process.

Web developers spend a ton of time into making their UI handle a slow backend well which is great and you should do that too (you don’t control network latency after all). But *****great***** web developers put effort into making it so their users hardly ever experience that great pending UI by making the backend as fast as possible. Your UI can’t be any faster than your slowest database query.

The Data Modeling Deep Dive Workshop will help you develop a keen understanding of databases and how to manage them. You'll learn the same tools and techniques used at the top tech companies to design robust database architectures and optimize query performance.

We'll be using production grade tools like Prisma and SQLite, but the concepts you learn in this workshop will be applicable to other database systems as well.

## Database Schema

A well-designed database schema is the foundation of your application. It's critical to get it right from the start, as it can be difficult to make changes down the road. This section provides a background on SQL before introducing Prisma's suite of tools. You'll learn:

- SQLite database initialization with Prisma
- Prisma's Object Relational Mapper (ORM)
- Defining a schema with Prisma's declarative schema language
- Converting a schema to SQL with the Prisma CLI
- Viewing and editing your database in Prisma Studio

## Relationships

Poorly planned relationships between database models can lead to data inconsistencies and increased query complexities and performance problems. To avoid these issues, you need to understand the different types of relationships and how to implement them. This section provides you with practical experience in recognizing relationships:

- One-to-many relationships (User has many Notes)
- One-to-one relationships (User has one UserImage)
- Updating the database with Prisma
- Planning models for the future
- Generate an Entity Relationship Diagram (ERD) for visualization

## Migrations

As your application evolves, so will your database schema. It's not uncommon for there to be "breaking changes" that affects one or more of your database models. This is where migrations come in.

Migrations allow you to track and synchronize schema changes across environments. Traditionally migrations have been tricky to manage, but this section will teach you how to use migrations to ensure your development, staging, and production environments are always in sync:

- Generating SQL migration code with Prisma
- Best practices for documenting & tracking migrations
- The "widen then narrow" approach for eliminating downtime
- Experimenting with migrations before permanently applying them

## Seeding Data

Manually adding data to a database is not only time-consuming but also prone to errors. Scripting to the rescue!

Here you'll learn how to populate your databases with consistent data for development and testing purposes:

- Initializing a SQLite database
- Idempotent scripting
- Resetting the database with Prisma
- Nesting SQL queries to simplify foreign key connections

## Generating Seed Data

Using hardcoded seed data works great for small applications, but it won't provide the best idea of how your application will perform at scale. You also shouldn't use real user data during development for plenty of obvious reasons.

A better approach is to use a library like Faker.js. This section will teach you how to use Faker.js to generate seed data for your database:

- Balancing realism with practicality when generating data
- Creating realistic user information
- Varying data quantities and randomness
- Validating against the schema
- Enforce data uniqueness

## Querying Data

In SQL, queries are written with the `SELECT` statement. This is one of the most powerful statements available, but it can also be one of the most complex. This is where tools like Object Relational Mappers (ORMs) come in, helping developers write SQL in a more intuitive way. Although it isn't a traditional ORM, the Prisma Client is an indispensable tool for modern web development. In this section, you'll learn how to get the most out of it:

- Understanding how Prisma's API maps to SQL
- Handling Hot Module Replacement (HMR)
- Logging slow queries
- Pre-connecting the client to the database

## Updating Data

Data mutation is a common task, but doing it incorrectly can lead to data corruption or loss. Especially when a single user action results in multiple database changes (for example, transferring money between user’s bank accounts). Getting mutations wrong can be a serious problem for your app’s success. This section will walk you through techniques for working safely:

- Deleting and updating data
- Conditional upserts
- Combining multiple queries into a transaction
- Rolling back to a previous transaction

## SQL

No matter how great an ORM is, sometimes you have to write raw SQL. In this section, you'll learn how to write raw SQL queries and execute them with Prisma. You'll also practice with:

- Wildcard queries
- Runtime schema validation with Zod
- Using joins to enable more complex queries
- Specifying query result ordering for more efficient searching

## Query Optimization

Nobody likes a slow application.

There are many stories of companies spending millions of dollars on scaling hardware, adding multiple levels of caching, and even rewriting their entire application in pursuit of a faster performance– only to find out later that the solution was optimizing their database indexes and queries.

In this final section, you'll learn how to optimize your queries for maximum performance:

- Identifying the index
- Analyzing raw SQL queries
- Multi-column indexing
- Balancing premature optimization with practicality

The Data Modeling Deep Dive Workshop will teach you best practices for designing database schemas, managing relationships, data mutations, and optimizations.

Data Modeling Deep Dive

Kent C. Dodds

Full Stack Vol 1

Intro to Data Modeling Deep Dive Workshop

Intro to Data Modeling Deep Dive

Intro to Database Schema 

Learn how to set up your database using Prisma, including initializing the schema and configuring new files.

Learn how to create a SQLite database using Prisma, define models with unique fields, and explore data using Prisma Studio. 

Database with Prisma and SQLite

Setting Up the Database with Prisma

Dad Joke Break Database Schema

Database Schema

Intro to Data Relationships

Create a one-to-many relationship: Users own multiple notes; note model handles the relationship, user model stays unaware. 

Learn how to establish one-to-many relationships between models in Prisma, including configuring relationship attributes and avoiding common pitfalls.

One-to-Many Relationships in Prisma

One-to-Many Relationships 

Optimize image storage in a DB: no polymorphism, future-proof for web apps. Learn why it's best for new projects.

Master Prisma relationships: one-to-one, one-to-many, foreign keys, cascading, unique constraints. Dive into DB schema design flexibilities & limits.

One-to-One and One-to-Many Relationships in Prisma

Database Design

Dad Joke Break Data Relationships

Data Relationships

Intro to Data Migrations

Master Prisma migration: create tables, modify props, spin up DBs, deploy across environments with confidence.

Prisma migrations: Seamlessly evolve your DB schema, reap benefits of flexibility and easy updates.

Database Migrations with Prisma

Creating Migration Files for Database Management with Prisma

Dad Joke Break Data Migrations

Data Migrations

Intro to Seeding Data

Configuring Prisma for seamless data seeding using seed scripts in your application.

Creating and Managing Data with Prisma Seed Scripts

Configuring Prisma for Seed Script Execution

Leverage Prisma nested mutations for efficient data handling, query optimization, and seamless retrieval. 

Efficiently manage relational records with nested queries. Explore benefits and considerations of nesting in DB ops.

Nested Queries in Database Operations

Optimizing Queries with Nested Mutations in Prisma

Dad Joke Break Seeding Data

Seeding Data

Intro to Generating Seed Data

Automatically generate realistic fake data for efficient testing with powerful tools.

Use Faker.js for realistic random user and note data, automating database population matching your schema. 

Generating Random User Data with Faker.js for Database Seeding

Generating Fake Data for Efficient Testing

Effortlessly generate dynamic web app data: user profiles, images, notes, with time-saving techniques.  

Create dynamic seed data with Faker and Prisma: random users, notes, images. Boost data generation efficiency and flexibility.

Generating Dynamic Seed Data 

Dynamically Generating Data

Enforce Unique: Seed user data with unique values, reduce collisions, and optimize queries.

Create seed data with unique valid usernames, enforce integrity, and safeguard against failures.

Generating Unique and Valid Usernames in Seed Data

Creating Unique User Data with Enforce Unique Library

Dad Joke Break Generating Seed Data

Generating Seed Data

Intro to Querying Data

Learn how to migrate from an in-memory database to a real database using Prisma and SQLite, enabling the loading of actual data in your application.

Optimize Prisma Client: Lazy connections, custom logging, and speed up app with query performance.

Optimizing Prisma Client for Efficient Database Operations

Connecting to a Real Database with Prisma and SQLite

Transition from in-memory to real database with Prisma, efficient SELECT queries for user data, type safety advantages.

Shift to Prisma SQLite DB, type-safe queries for specific fields, end-to-end type safety in web apps. 

Using Prisma to Retrieve User Data from a SQLite Database

Transitioning to Real Database with Prisma API

Learn how to optimize note loading by combining multiple queries into a single query and migrating to Prisma for real database integration.

Optimize data retrieval with Prisma, subselects for efficiency, TypeScript error-catching, and migration guidance. 

Efficient Data Retrieval with Prisma and Subselects

Database Queries and Migrating to Prisma 

Dad Joke Break Querying Data

Querying Data

Intro to Updating Data

Delete data from the actual database using Prisma. This exercise introduces a straightforward mutation before diving into more complex concepts.

Discover how easy it is to delete data using Prisma's syntax and APIs. 

Deleting Data with Prisma

Mutating Data with Prisma

Mutations for updating images and notes on a page, including cache control considerations and utilizing helpful utilities for easier data manipulation.

Learn how to update and delete data in a form using Prisma, while handling image updates, removing deleted images, and creating new images. 

Efficiently Updating and Deleting Data in a Form with Prisma

Updating Page Mutations for Images and Notes with Efficient Cache Control

Learn how to use transactions to ensure that data operations within your application either succeed completely or fail entirely. 

Learn how to use prisma.transaction to ensure a group of database operations either all succeed or all fail together

Implementing Transactions with Prisma for Atomic Database Operations

Implementing Transactions for Reliable Data Operations

Learn how to use nested queries in Prisma to combine multiple database operations into a single optimized query, reducing the number of calls made to the db. 

Learn how to optimize database updates using Prisma by combining multiple related queries into a single transaction, resulting in cleaner, more efficient code. 

Efficient Database Updates with Prisma's Nested Queries

Optimizing Database Calls with Nested Queries in Prisma

Dad Joke Break Updating Data

Updating Data

Intro to SQL

Master advanced user search with Prisma: Write raw SQL queries, iterate from basic search to SQLite database display.

Search users with Prisma raw queries by criteria, handle TypeScript hurdles, prep for adding images.

User Search with Prisma and SQL

SQL Queries for User Search in Prisma

Tackle TypeScript errors in Prisma raw queries, ensure type safety with Zod runtime checks. Weigh Prisma's trade-offs.

Embrace runtime type checks in Prisma for robust DB interaction, handle schema changes and parsing errors.

Runtime Type Checking and Parsing in Prisma with Raw Queries

Handling TypeScript Errors and Runtime Type Checking with Prisma and Zod

Efficiently combine data with SQL joins: inner, left, right, full. Get user profile photos without performance issues.

Master left joins in Prisma/SQLite, resolve conflicts, enrich schema, and improve UI with extra properties.

Left Joins with Prisma and SQLite

Working with Joins in SQL

Sort users by most recent note updates using Prisma raw SQL queries. Follow step-by-step instructions.

AI assistants like GitHub Copilot for SQL queries: benefits, best practices, code review, productivity, accuracy. 

Leveraging AI Assistance for Writing SQL Queries

Sorting Users by Recent Entry with Raw SQL

Dad Joke Break SQL

Intro to Query Optimization

Enhance slow user search with single index in SQLite, sidestep caching complexities, grasp query plans, and boost DB performance.

Learn how to analyze query plans and optimize query performance by adding indexes to improve speed and avoid unnecessary scans and temporary data structures.

Data Modeling Deep Dive

Data Modeling Deep Dive

Transcript

Optimizing Query Performance with Indexes in Prisma

Transcript