Working With MongoDB Data in Nodejs Apps With Mongoose
Working With MongoDB Data in Nodejs Apps With Mongoose
This is something that’s not available with the native MongoDB client.
The object has the fields as the keys and the data type as the values.
Then we use the Kitten constructor, and then we call save to save the
Kitten object to the database.
kittySchema.methods.speak = function () {
console.log(`hello ${this.name}`);
}
kittySchema.methods.speak = function () {
console.log(`hello ${this.name}`);
}
We can’t autodetect and save changes when we use this type since it’s a
schema-less type.
ObjectIds
We can specify the ObjectId type to store object IDs.
Then we can check the types of car.driver . We should get the type is
'object' .
The 2nd console log should be true since driver is of type ObjectId .
Boolean
Another type that we set to fields is the boolean type.
true
'true'
1
'1'
'yes'
false
'false'
0
'0'
'no'
To do that, we write:
const mongoose = require('mongoose');
const connection = "mongodb://localhost:27017/test";
mongoose.connect(connection, { useNewUrlParser: true });
const db = mongoose.connection;
db.on('error', () => console.error('connection error:'));
db.once('open', () => {
console.log('connected')
});
We connect to the server with the URL and the test collection name.
Operation Buffering
We can start using the models immediately without waiting fir Mongoose to
establish a connection.
So we can write:
const mongoose = require('mongoose');
const connection = "mongodb://localhost:27017/test";
mongoose.connect(connection, { useNewUrlParser: true });
const db = mongoose.connection;
db.on('error', () => console.error('connection error:'));
db.once('open', () => {
console.log('connected')
});
We create our model and use it without waiting for the connect to complete
since we have the database code outside the callback.
or:
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test', {
useNewUrlParser: true }).
catch(error => handleError(error));
to connect to a database.
We can also listen to the error event as we did in the example above.
Callback
The mongoose.connect method takes a callback to so we can check for
errors.
then they all have a any field with the mixed type.
Maps
Mongoose version 5.1.0 or later has the Map type to store key-value pairs.
Then we can pass in the object to specify the key-value pairs for the map.
And we can get the value by the key with the get method.
Getters
We can add getters to our schema fields.
By default the picture property has the return value of the getter.
But we can also set the getters property to false to get the picture field’s
value.
SchemaType
A SchemaType is a configuration object for an individual property.
If we have nested fields in a schema, we also have to set the types for the
nested fields:
const mongoose = require('mongoose');
const connection = "mongodb://localhost:27017/test";
mongoose.connect(connection, { useNewUrlParser: true });
const db = mongoose.connection;
db.on('error', () => console.error('connection error:'));
db.once('open', () => {
console.log('connected')
});
They include:
We control how the value is get and set with the get and set methods
respectively.
to add the test schema with the index property set to true to add an index
for the test field.
They include:
min: Number, creates a validator that checks if the value is greater than
or equal to the given minimum.
max: Number, creates a validator that checks if the value is less than or
equal to the given maximum.
enum: Array, creates a validator that checks if the value is strictly equal
to one of the values in the given array.
Date Schema Types
For date schema types, we can set:
min: Date
max: Date
We called mongoose.Schema that has the name and size string fields.
Then we create the model with the schema with the mongoose.model
method.
Next, we use the model class to create the document with the data we want.
The callback we pass into the save method has the err parameter that will
be defined if there’s an error.
We can also add the create static method to create the document:
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test')
const schema = new mongoose.Schema({ name: 'string', size:
'string' });
const Tank = mongoose.model('Tank', schema);
Tank.create({ size: 'small' }, (err, small) => {
if (err) {
return console.log(err);
}
console.log(small);
});
And the 2nd argument is the callback that’s called when the result is
computed.
Also, we can call the insertMany static method on the Tank model:
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test')
const schema = new mongoose.Schema({ name: 'string', size:
'string' });
const Tank = mongoose.model('Tank', schema);
Tank.insertMany([{ size: 'small' }], (err) => {
if (err) {
console.log(err);
}
});
We call find with the key and value we’re looking for.
The where method has the field that we want to search for.
exec runs the query, and the callback has the results.
Deleting
We can delete items with the deleteOne method:
const mongoose = require('mongoose');
const connection =
mongoose.createConnection('mongodb://localhost:27017/test');
const schema = new mongoose.Schema({ name: 'string', size:
'string' });
const Tank = connection.model('Tank', schema);
Tank.deleteOne({ size: 'large' }, (err) => {
if (err) {
return console.log(err);
}
});
We delete the first item that has the size field equal to 'large' .
Updating
We can update the first with the given key and value with the updateOne
method.
The 2nd argument is the key-value pair we want to update the document
with.
The 3rd argument is the callback that’s called after the operation is done.
Documents
Mongoose documents are one to one mapping to documents as stored in
MongoDB.
It’s also an instance of the Model constructor and the Document constructor.
Retrieving
When we retrieve a document with findOne , it returns a promise that
resolves to a document.
We see again that all 3 console logs are true , so we know that documents
are retrieved with findOne .
Updating
We can modify a document and call save to update a document.
Then we set the property to the given value and the call save to save the
changes.
If the document with the given _id isn’t found, then we’ll get a
DocumentNotFoundError raised:
to add the validator method to our method that returns a promise instead
of a boolean directly.
Then we can use the validate method to validate the values we set.
And then we can catch validation errors with the catch block.
We can get the message from the errors property in the error object.
Validation Errors
Errors returned after validation has an errors object whose values are
ValidatorError objects.
If an error is thrown in the validator, the property will have the error that
was thrown.
The name value also has a validator added to it by passing a callback into
the validate method to validate the name field.
Cast Errors
Mongoose tries to coerce values into the correct type before validators are
run.
If data coercion fails, then the error.errors object will have a CastError
object.
to call the overwrite method to change the first tanks document with the
name field set to 'James' .
Subdocuments
Subdocuments are documents that are embedded in other documents.
We created the Child and Parent schemas with the Child schema
embedded in the Parent schema.
The children property has an array of documents that fie the Child
schema.
When we call save on the parent , everything embedded inside will also be
saved.
We can watch the validate and save events for each schema.
They should run in the order in the same order the numbers are in.
The 2nd entry of the min array has the error message.
The drink field has more validation. We have the required method to
check other fields to make this field required only if this.bacon is bigger
than 3.
to get the error as we did in the last line of the run function.
The unique option isn’t a validator. It lets us add unique indexes to a field.
We add the email field to with the validate method with the validator
function to add validation for the email field.
We get the children subarray’s first entry’s name property and set it to
'Mary' .
Then we save the parent and child entries with the save method.
Adding Subdocuments to Arrays
We can add subdocuments to arrays.
We can use the findOne method to return the first entry that matches the
query.
We create the Person schema and save a document that’s created from the
Person constructor.
The 2nd argument is a string with the columns that we want to select.
Even though we can use async and await , with the findOne method, it
doesn’t return a promise.
Then then method is provided so that we can use the async and await
syntax.
Then we use the while loop to get the cursor.next() method to get the
next item from the cursor object.
We don’t need the cursor method anymore since find and other query
methods return the cursor when we use it with for-await-of .
Cursor Timeout
We can set the noCursorTimeout flag to disable cursor timeout.
This will return plain JavaScript objects rather than Mongoose documents.
Then we use a for-of loop to loop through the items since it returns a
regular JavaScript array.
The aggregate method doesn’t cast its pipeline, so we have to make sure
the values we pass in the pipeline have the correct type.
Model.deleteMany()
Model.deleteOne()
Model.find()
Model.findById()
Model.findByIdAndDelete()
Model.findByIdAndRemove()
Model.findByIdAndUpdate()
Model.findOne()
Model.findOneAndDelete()
Model.findOneAndRemove()
Model.findOneAndReplace()
Model.findOneAndUpdate()
Model.replaceOne()
Model.updateMany()
Model.updateOne()
Then both console log statements are true because the parent method
returns the parent object of the child and a subdocument in the children
fields.
One way is to put the array straight into the schema object:
async function run() {
const { createConnection, Schema } = require('mongoose');
const connection =
createConnection('mongodb://localhost:27017/test');
const parentSchema = new Schema({
children: [{ name: 'string' }]
});
const Parent = await connection.model('Parent',
parentSchema);
const doc = new Parent({ children: { name: 'test' } });
await doc.save();
}
run();
or:
async function run() {
const { createConnection, Schema } = require('mongoose');
const connection =
createConnection('mongodb://localhost:27017/test');
const schema = new Schema({
nested: {
type: new Schema({ prop: String }),
required: true
}
});
const Parent = await connection.model('Parent', schema);
}
run();
Both ways will set the nested subdocument with the data type string.
Required Validators on
Nested Objects
We can define validators on nested objects with Mongoose.
Now when we create a new Person instance, we’ll see an error because we
haven’t added those properties into our document.
Update Validators
Mongoose also supports validation for updating documents with the
update , updateOne , updateMany , and findOneAndUpdate methods.
Since we have the runValidators property set to true in the opts object,
we’ll get validator when we call the updateOne method.
Then we should see the ‘Invalid color’ message logged in the console log in
the callback.
Update Validators and this
The value of this for update validators and document validators.
The context optioin lets us set the value of this in update validators.
This is because Mongoose calls the pre('save') hook that calls validate .
Then they’ll be called one by one in the same order that they’re listed.
Query Middleware
Pre and post save hooks aren’t run when update methods are run.
Then when we have query set to false or didn’t add the query property,
then the updateOne pre hook won’t run when we run updateOne .
Aggregation Hooks
We can add aggregation hooks.
We listen to the update event and get the error from the error parameter.
We can get the name and code to get information about the error.
Synchronous Hooks
Some hooks are always synchronous.
Now when we retrieve the latest value of the story with the author with
populate , we’ll see that story is null .
Field Selection
We can select a few fields instead of selecting all the fields when we call
populate .
Then we call findOne to get the story document with the author and the
name properties.
Populating Multiple Paths
If we call populate multiple times, then only the last one will take effect.
Query Conditions and
Other Options
The populate method can accept query conditions.
Then when we call populate with an object that finds all the fans with age
greater than or equal to 21, we should get our fan entry.
The select property has a string that lets us select the name field but not the
_id as indicated by the - sign before the _id .
then only the second updateOne callback will have its err parameter
defined because we unset the name field when it’s required.
$set
$unset
$push (>= 4.8.0)
$addToSet (>= 4.8.0)
$pull (>= 4.12.0)
$pullAll (>= 4.12.0)
Middleware
Middlewares are functions that are passed control during the execution of
async functions.
validate
save
remove
updateOne
deleteOne
init (init hooks are synchronous)
Query middleware are supported for some model and query functions. They
include:
count
deleteMany
deleteOne
find
findOne
findOneAndDelete
findOneAndRemove
findOneAndUpdate
remove
update
updateOne
updateMany
They are run one after the other by each middleware calling next .
to add a pre middleware for the that runs before the save operation.
Once the next function is run, the save operation will be done.
If we call next in our callback, calling next doesn’t stop the rest of the
middleware code from running.
We should add return to stop the code below the if block from running:
async function run() {
const { createConnection, Schema } = require('mongoose');
const connection =
createConnection('mongodb://localhost:27017/test');
const kittenSchema = new Schema({
name: { type: String, required: true },
age: Number
});
kittenSchema.pre('save', async () => {
if (Math.random() < 0.5) {
console.log('calling next');
return next();
}
console.log('after next');
});
const Kitten = connection.model('Kitten', kittenSchema);
}
run();
Populate
We can use the populate method to join 2 models together.
author.save(function (err) {
if (err) return handleError(err);
story1.save(function (err) {
if (err) {
return console.log(err);
}
Story.
findOne({ title: 'Mongoose Story' }).
populate('author').
exec(function (err, story) {
if (err) {
return console.log(err);
}
console.log('author', story.author.name);
});
});
});
}
run();
The Story model references the Person model by setting author._id as the
value of the author field.
Then in the callback for story1.save , we get the Story entry with the
exec method with a callback to get the data.
Then we can access the author’s name of the story with the
story.author.name property.
The documents created with the models will be saved in their own
collections.
author.save(function (err) {
if (err) return handleError(err);
story1.save(function (err) {
if (err) {
return console.log(err);
}
author.save(function (err) {
if (err) return handleError(err);
story1.save(function (err) {
if (err) {
return console.log(err);
}
author.save(function (err) {
if (err) return handleError(err);
story1.save(function (err) {
if (err) {
return console.log(err);
}
We added post middlewares for the init , validate , save , and remove
operations that are run after the given document operations.
Async Post Hooks
We can add async post hooks. We just need to call next to proceed to the
next post hook:
async function run() {
const { createConnection, Schema } = require('mongoose');
const connection =
createConnection('mongodb://localhost:27017/test');
const schema = new Schema({
name: { type: String, required: true },
age: Number
});
schema.post('save', (doc, next) => {
setTimeout(() => {
console.log('post1');
next();
}, 10);
});
We call the next function in the setTimeout callback to proceed to the next
post middleware.
Define Middleware Before
Compiling Models
We have to define middleware before we create the model with the schema
for it to fire.
We created the schema and then added a save pre middleware right after we
defined the schema and before the model is created.
This way, the callback in the pre method will be run when we create a
document with the model.
limit vs. perDocumentLimit
Populate has a limit option, but it doesn’t support limit on a per-document
basis.
Now when we query the story, we get the fans array with the fan in it.
Dynamic References via refPath
We can join more than one model with dynamic references and the refPath
property.
The subject is set to an object ID. We have the refPath property that
references the model that it can reference.
Then we call populate with subject to get the subject field’s data.
Equivalently, we can put the related items into the root schema.
Then we call populate on both fields so that we can get the comments.
Populating an Existing Document
We can call populate on an existing document.
We created the story with the fans and author field populated.
Then we call populate in the resolved story object and then call
execPopulate to do the join.
Now the console log should see the fans entries displayed.
Populating Multiple Existing
Documents
We can populate across multiple levels.
We can create the Conversation and Event entries and link them together.
And then we can call findOne on Event to get the linked data.
Discriminators
Discriminators are a schema inheritance mechanism.
const clickedEvent =
new ClickedLinkEvent({ time: Date.now(), url: 'mongodb.com'
});
console.log(clickedEvent)
}
run();
We get:
{ _id: 5f6f78f17f83ca22408eb627, time: 2020-09-26T17:22:57.690Z
}
And the rest of the models are created from the Event.discriminator
method.
Then we created the models and saved them all.
to get the type of data that’s saved from the console logs. We should get:
undefined
ClickedLink
SignedUp
logged.
We can add the discriminatorKey in the options to change the
discriminator key.
So we can write:
async function run() {
const { createConnection, Types, Schema } =
require('mongoose');
const db =
createConnection('mongodb://localhost:27017/test');
const options = { discriminatorKey: 'kind' };
const eventSchema = new Schema({ time: Date }, options);
const Event = db.model('Event', eventSchema);
to set the options for each model and then access the kind property instead
of __t and get the same result as before.
Embedded Discriminators
in Arrays
We can add embedded discriminators into arrays.
Next, we create the Batch model from the batchSchema so that we can
populate the model.
We call Batch.create with the batch object that has an events array
property to add the items.
Then we use as the schema for the objects in the events array property.
Then we create the SubEvent model calling the path and the
discriminator methods.
call.
Now we have can subEvents properties in events array entrys that are
recursive.
Next, we create the list object with the events array that has the
subEvents property added recursively.
The last 2 console logs should get the subevents from the 2nd event entry.
Discriminators and Queries
Queries are smart enough to take into account discriminators.
And from the console log, we can see that both the _id and time fields of
event1 are strings.
So the _id field is the same one as the eventSchema , but the
ClickedLinkEvent field has the same type as the one in
clickedLinkSchema .
Using Discriminators with
Model.create()
We can use discriminators with the Model.create method.
const shapes = [
{ name: 'Test' },
{ kind: 'Circle', radius: 5 },
{ kind: 'Square', side: 10 }
];
const [shape1, shape2, shape3] = await Shape.create(shapes);
console.log(shape1 instanceof Shape);
console.log(shape2 instanceof Circle);
console.log(shape3 instanceof Square);
}
run();
Then in the console log, they should all log true since we specified the type
of each entry.
BandSchema.virtual('members', {
ref: 'Person',
localField: 'name',
foreignField: 'band',
justOne: false,
options: { sort: { name: -1 }, limit: 5 }
});
BandSchema.virtual('members', {
ref: 'Person',
localField: 'name',
foreignField: 'band',
justOne: false,
options: { sort: { name: -1 }, limit: 5 }
});
const Person = db.model('Person', PersonSchema);
const Band = db.model('Band', BandSchema);
const person = new Person({ name: 'james', band: 'superband'
});
await person.save();
const band = new Band({ name: 'superband' });
await band.save();
const bands = await Band.find({}).populate('members').exec();
console.log(bands[0].members);
}
run();
BandSchema.virtual('members', {
ref: 'Person',
localField: 'name',
foreignField: 'band',
justOne: false,
options: { sort: { name: -1 }, limit: 5 }
});
We call populate with an object with the path to get the virtual field and
the select property has a string with the field names that we want to get
separated by a space.
Populate Virtuals: The
Count Option
We can add the count option when we create a populate virtual to get the
count of the number of children we retrieved.
BandSchema.virtual('numMembers', {
ref: 'Person',
localField: 'name',
foreignField: 'band',
count: true
});
The ref property is the model that the Band model is referencing.
count set to true means that we get the count. numMembers is the field name
that we get the count from.
Then we save a Person and Band document with the same name.
Then we call populate with the numMembers field to get the number of
members.
And finally, we get the numMembers field from the retrieved result to get
how many Person children entries are in the Band .
Populate in Middleware
We can add pre or post hooks to populate operations.
BandSchema.virtual('numMembers', {
ref: 'Person',
localField: 'name',
foreignField: 'band',
count: true
});
BandSchema.pre('find', function () {
this.populate('person');
});
to add the pre and post hooks to listen to the find operation.
We should always call populate with the given field to do the population.
Discriminators and Single Nested
Documents
We can define discriminators on single nested documents.
Then we use them by setting the kind property when we create the entry.
Plugins
We can add plugins to schemas.
Plugins are useful for adding reusable logic into multiple schemas.
We call schema.post in the plugin to listen to the find and findOne events.
Then we loop through all the documents and set the loadedAt property and
set that to the current date and time.
In the run function, we add the plugin by calling the plugin method on
each schema.
Now we should see the timestamp when we access the loadedAt property
after calling findOne .
Conclusion
To make manipulating Mongodb data easier in Node.js apps, we can use the
Mongoose library.
It comes with features like schemas and models to restrict the data types
that can be in a document.
We can also create models that inherit from other models with
discriminators.