DZ Mongodb Essentials 2024
DZ Mongodb Essentials 2024
DZ Mongodb Essentials 2024
CONTENTS
• Configuration Options
• Using the Shell
SETUP OPTIONS
Startup options for MongoDB can be set on the command line or in a
configuration file. The syntax is slightly different between the two:
Table 1
--auth auth=true
the mongo shell was deprecated in MongoDB v5.0 and removed from what errors it can throw, as well as how to run it from another language.
There are a number of functions that give you a little help if you forget in. The shell helper edit makes this easier, which opens up a text
a command: editor, allowing you to edit variables from there. For example:
• DBCollection.prototype.drop
Using the opid field from above, you can kill operations: • explainVersion is the output format version.
• command is the command being explained.
> db.killOp(123)
• queryPlanner provides information about the
selected and rejected plans by the query optimizer.
Not all operations can be killed or will be killed immediately. In
general, operations that are waiting for a lock cannot be killed until • executionStats provides execution details of the
they acquire the lock. accepted and rejected plans.
• serverInfo provides information about the MongoDB instance. To see a member's view of the entire set, connect to it and run the
• serverParameters provides details about the following command:
internal parameters. > rs.status()
TYPES OF CURSORS
This command returns a structured JSON output and shows you
Here are some common cursor types in MongoDB:
what it thinks the state and status of the other members are. Running
• Standard cursor is the default type returned by rs.status() on a secondary node will show you which node the
db.collection.find(). It iterates over query results secondary is syncing from in the syncSourceHost field.
in batches, retrieving data on demand from the server.
SHARDING
• Change Stream cursor is a real-time data monitor, notifying
To see your cluster's metadata (shards, databases, chunks, etc.),
you whenever a document in a collection is inserted,
execute the following command from the MongoDB shell (mongosh)
updated, deleted, or replaced.
connected to any member of the sharded cluster:
• Tailable cursor is a cursor for a capped collection that remains
> db.printShardingStatus()
open after the client exhausts the results in the initial cursor.
• Backup cursor is a type of tailable cursor that points to a If verbosity is set to true, it displays full details of the chunk distribution
list of backup files. Backup cursors are for internal use only.
across shards along with the number of chunks on each shard:
• Orphaned cursor is a cursor that is not correctly
> db.printShardingStatus(true)
closed or iterated over in your app code. It can cause
performance issues.
sh.status can also be executed on a mongos instance to fetch sharding
> db.foo.find().hint({x:1})
You can also connect to the mongos and see data about your shards,
databases, collections, or chunks by using use config, then querying
SYSTEM PROFILING
the relevant collections:
You can turn on system profiling to see operations currently happening
on a database. Note that there is a performance penalty to profiling, > use config
switched to db config
but it can help isolate slow queries.
> show collections
changelog
> db.setProfilingLevel(2) // profile all operations
chunks
> db.setProfilingLevel(1) // profile operations
collections
that take longer than 100ms
csrs.indexes
> db.setProfilingLevel(1, 500) // profile
databases
operations that take longer than 500ms migrationCoordinators
> db.setProfilingLevel(0) // turn off profiling mongos
> db.getProfilingLevel(1) // see current profiling rangeDeletions
setting settings
shards
Profile entries are stored in a capped collection called system.profile tags
version
within the database in which profiling was enabled. Profiling can be
turned on and off for each database.
Always connect to a mongos to get sharding information. Never connect
REPLICA SETS or write directly to a config server; always use sharding commands
To find replication lag information for each secondary node, connect to and helpers.
the primary node of the replica set and run this command:
After maintenance, sometimes mongos processes that were not
> rs.printSecondaryReplicationInfo() actually performing the maintenance will not have an updated
source: m1.demo.net:27002
version of the config. Either bouncing these servers or running the
syncedTo: Mon Feb 01 2023 10:20:40 GMT-0800
flushRouterConfig command is generally a quick fix to this issue:
(PST)
20 secs (0 hrs) behind the primary
> use admin
> db.runCommand({flushRouterConfig:1})
The above command prints a formatted output of the replica set
status. You can also use db.printReplicationInfo() to retrieve
Often this problem will manifest as setShardVersion failed errors.
the replica set member's oplog. Its output is identical to that of
rs.printReplicationInfo().
Don't worry about setShardVersion errors in the logs, but they The symbols in Table 4 indicate the following:
should not trickle up to your application. Note that you shouldn't get • ✓ = matches
the errors from a driver unless the mongos it's connecting to cannot • X = does not match
reach any config servers.
Table 4
INDEX OPTIONS OPERATOR EXAMPLE QUERY RESULT
The table below provides several index options. For a complete list,
$gt {numSold : ✓ {numSold: 1}
refer to: https://www.mongodb.com/docs/manual/reference/method/ $gte {$lt:3}}
X {numSold: "hello"}
$lt
db.collection.createIndex
$lte X {x : 1}
$ne
Table 3 $in {age : {$in : ✓ {age: 21}
$nin [10, 14, 21]}}
INDEX OPTION DESCRIPTION ✓ {age: [9, 10, 11]}
X {age: 9}
unique Creates a unique index on a collection to
prevent insertion or update of documents $all {hand : {$all : ✓ {hand: ["7", "8", "9",
["10","J","Q", "10", "J", "Q", "K", "A"]}
with duplicate index key values; it is false "K","A"]}}
by default. X {hand:["J","Q","K"]}
$not { $nor: [{ ✓ { "status": "active",
name If not specified, MongoDB generates the status: "active" "age": 70 }
index name by concatenating the names of }, { age: { $gte:
X { "status": "inactive",
indexed fields and the sort order. 65 } }] }
"age": 45 }
partialFilterExpression If specified, the index will only reference $mod {age : {$mod : ✓ {age: 50}
documents that match the provided filter [10, 0]}}
X {age: 42}
expression.
$exists {phone: {$exists: ✓ {phone: "555-555-5555"}
sparse If set to true, the index will only reference true}}
X {phones: ["555-555-5555",
documents with the specified field; it is "1-800-555-5555"]}
false by default.
$type* {age : {$type : ✓ {age : "42"}
expireAfterSeconds Time to live (in seconds) that controls 2}}
X {age : 42}
how long MongoDB retains documents in
$size {"top- ✓ {"top-three":["gold","sil
this collection. three":{$size:3}} ver","bronze"]}
hidden Controls whether the index is hidden from X {"top-three":["blue
the query planner. ribbon"]}
$regex {role: / ✓ {"top-three":["gold","sil
storageEngine Specifies storage engine during index admin.*/i} {role: ver","bronze"]}
creation. {$regex:'admin.*',
X {"top-three":["blue
$options: 'i' }}
ribbon"]}
$not { $nor: [{ status: ✓ { "status": "active",
QUERY OPERATORS "active" }, { age: "age": 70 }
Queries are generally of the form: { $gte: 65 } }] }
X { "status": "inactive",
"age": 45 }
{key : {$op : value}}
$all { genres: { $all: ✓ {
["fiction", "title": "The Da Vinci
"mystery"] } } Code",
For example:
"genres": ["fiction",
"mystery", "thriller"]
{age : {$gte : 18}} }
X {
There are three exceptions to this rule — $and, $or, and $nor — which "title": "Harry Potter
and the Sorcerer's Stone",
are all top level: "genres": ["fantasy",
"adventure"]
{$or : [{age: {$gte : 18}}, {age : {$lt : 18}, }
UPDATE OPERATORS
Table 5 includes commonly used MongoDB update operations:
Table 5
AGGREGATION PIPELINE OPERATORS $project and $group can both take expressions, which can use the
The aggregation framework can be used to perform everything $fieldName syntax as shown below:
Table 6 contains list of operators for the available stages: $mod : ["$sum", The remainder of dividing sum by count.
"$count"]
Table 6 $multiply : ["$mph", Multiplies mph by 24*365.
24, 365]
OPERATOR DESCRIPTION
$subtract : ["$price", Subtracts discount from price.
{$project : projection} Includes, excludes, renames, and "$discount"]
munges fields
$strcasecmp : ["ZZ", 1 if name is less than ZZ, 0 if name is ZZ,
{$match : match} Queries and takes an argument identical "$name"] -1 if name is greater than ZZ.
to that passed to find()
$substr : ["$phone", Gets the area code (first three
{$limit : num} Limits results to num 0, 3] characters) of phone.
{$skip : skip} Skips num results $toLower : "$str" Converts str to all lowercase.
{$sort : sort} Sorts results by the given fields $toUpper : "$str" Converts str to all uppercase.
{$group : group} Groups results using the expressions $ifNull : If mightExist is not null, it returns
["$mightExist", $add : mightExist. Otherwise, it returns the
given (see Table 7)
["$doesExist", 1]] result of the second expression.
{$unwind : field} Explodes an embedded array into its own
$cond : [exp1, exp2, If exp1 evaluates to true, it returns
top-level documents
exp3] exp2. Otherwise, it returns expr3.
1. Use the fsyncLock() command, which flushes all in-flight writes to To prevent a secondary from being elected temporarily, connect to it
disk and prevents new ones: and issue the freeze command:
> db.fsyncUnlock()
STARTING A MEMBER AS A STAND-ALONE SERVER
{ info: 'fsyncUnlock completed', lockCount:
For maintenance, often, it is desirable to start up a secondary and be
Long('0'), ok: 1
able to do writes on it (e.g., for building indexes). To accomplish this,
you can start up a secondary as a stand-alone mongod temporarily.
Note: To restore from this backup, copy the files to the correct server's
dbpath and start the mongod.
If the secondary was originally started with the following arguments:
Alternatively, if you have a filesystem that does filesystem $ mongod --dbpath /data/db --replSet setName --port
snapshots, your journal is on the same volume, and you haven't done 30000
anything stripy with RAID, you can take a snapshot without locking. In
this case, when you restore, the journal will replay operations to make Then shut it down cleanly and restart it with:
the data files consistent.
$ mongod --dbpath /data/db --port 30001
There are several other options for backing up your MongoDB data:
Note that the dbpath does not change but the port does, and the
• Percona Backup for MongoDB (PBM) – An open-source and replSet option is removed (all other options can remain the same).
distributed solution for making consistent backups and This mongod will come up as a stand-alone server. The rest of the
restoring MongoDB sharded clusters and replica sets. You can replica set will be looking for a member on port 30000, not 30001, so
either use the command-line interface for backups on a running it will just appear to be "down" to the rest of the set.
server or manage backups from a web interface with PBM and
Percona Monitoring and Management. When you are finished with maintenance, restart the server with the
original arguments.
• mongodump and mongorestore – mongodump is used to create a
binary export of MongoDB data, while mongorestore is used to USER MANAGEMENT
import this data back into a MongoDB instance. To check current user privileges:
To revoke a role:
> db.revokeRolesFromUser(
UPDATED BY ABHISHEK GUPTA,
... "sensorsUserAdmin", PRINCIPAL DEVELOPER ADVOCATE, AWS
... [
Over the course of his career, Abhishek has worn
... { role: "userAdmin", db: "sensors" }
multiple hats including engineering, product
... ] management, and developer advocacy. Most of his
... ) work has revolved around open-source technologies,
including distributed data systems and cloud-native platforms.
Abhishek is also an open source contributor and avid blogger.
MONGODB RESTRICTIONS
Below are common limitations in MongoDB. For a full list, see
https://www.mongodb.com/docs/manual/reference/limits.
3343 Perimeter Hill Dr, Suite 100
• The maximum document size is 16 megabytes. Nashville, TN 37211
888.678.0399 | 919.678.0300
• The index entry total size must be less than 1,024 bytes.
At DZone, we foster a collaborative environment that empowers developers and
tech professionals to share knowledge, build skills, and solve problems through
• A collection can have up to 64 indexes. content, code, and community. We thoughtfully — and with intention — challenge
the status quo and value diverse perspectives so that, as one, we can inspire
• The index name (namespace included) cannot be longer than positive change through technology.
127 bytes (for version 4.0 or earlier).
Copyright © 2024 DZone. All rights reserved. No part of this publication may be
• A replica set can have up to 50 members.
reproduced, stored in a retrieval system, or transmitted, in any form or by means
of electronic, mechanical, photocopying, or otherwise, without prior written
• A shard key can have 512 bytes at most (for version 4.2 or earlier). permission of the publisher.
• A shard key is always immutable (for version 4.2 or earlier).