Transform Your Data with Aggregation

Compare Aggregation and Find Operations

The following table lists the different tasks that find operations can perform and compares them to what aggregation operations can perform. The aggregation framework provides expanded functionality that allows you to transform and manipulate your data.

Limitations

Consider the following limitations when performing aggregation operations:

• Returned documents cannot violate the BSON document size limit of 16 megabytes.

• Pipeline stages have a memory limit of 100 megabytes by default. You can exceed this limit by passing a value of true to the allowDiskUse() method and chaining the method to aggregate().

• The $graphLookup operator has a strict memory limit of 100 megabytes and ignores the value passed to the allowDiskUse() method.

Filter, Group, and Count Documents

This code example produces a count of the number of bakeries in each borough of New York. To do so, it calls the aggregate() method and passes an aggregation pipeline as a list of stages. The code builds these stages by using the following Aggregates helper methods:

• filter(): Builds the $match stage to filter for documents that have a cuisine value of "Bakery"

• group(): Builds the $group stage to group the matching documents by the borough field, accumulating a count of documents for each distinct value

val pipeline = Seq(Aggregates.filter(Filters.equal("cuisine", "Bakery")),
                   Aggregates.group("$borough", Accumulators.sum("count", 1))
)
collection.aggregate(pipeline)
          .subscribe((doc: Document) => println(doc.toJson()),
                    (e: Throwable) => println(s"There was an error: $e"))

{"_id": "Brooklyn", "count": 173}
{"_id": "Queens", "count": 204}
{"_id": "Bronx", "count": 71}
{"_id": "Staten Island", "count": 20}
{"_id": "Missing", "count": 2}
{"_id": "Manhattan", "count": 221}

Explain an Aggregation

To view information about how MongoDB executes your operation, you can instruct the MongoDB query planner to explain it. When MongoDB explains an operation, it returns execution plans and performance statistics. An execution plan is a potential way in which MongoDB can complete an operation. When you instruct MongoDB to explain an operation, it returns both the plan MongoDB executed and any rejected execution plans by default.

To explain an aggregation operation, chain the explain() method to the aggregate() method. You can pass a verbosity level to explain(), which modifies the type and amount of information that the method returns. For more information about verbosity, see Verbosity Modes in the MongoDB Server manual.

The following example instructs MongoDB to explain the aggregation operation from the preceding Filter, Group, and Count Documents example. The code passes a verbosity value of ExplainVerbosity.EXECUTION_STATS to the explain() method, which configures the method to return statistics describing the execution of the winning plan:

val pipelineToExplain = Seq(Aggregates.filter(Filters.equal("cuisine", "Bakery")),
                   Aggregates.group("$borough", Accumulators.sum("count", 1))
)
collection.aggregate(pipelineToExplain)
          .explain(ExplainVerbosity.EXECUTION_STATS)
          .subscribe((doc: Document) => println(doc.toJson()),
                    (e: Throwable) => println(s"There was an error: $e"))

{"explainVersion": "2", "queryPlanner": {"namespace": "sample_restaurants.restaurants",
"indexFilterSet": false, "parsedQuery": {"cuisine": {"$eq": "Bakery"}}, "queryHash": "865F14C3",
"planCacheKey": "0FC225DA", "optimizedPipeline": true, "maxIndexedOrSolutionsReached": false,
"maxIndexedAndSolutionsReached": false, "maxScansToExplodeReached": false, "winningPlan":
{"queryPlan": {"stage": "GROUP", "planNodeId": 3, "inputStage": {"stage": "COLLSCAN",
"planNodeId": 1, "filter": {"cuisine": {"$eq": "Bakery"}}, "direction": "forward"}},
...}

Search Operator Helper Methods

The Scala driver provides helper methods for the following operators:

{
  "mappings": {
    "dynamic": true,
    "fields": {
      "title": {
        "analyzer": "lucene.keyword",
        "type": "string"
      },
      "genres": {
        "normalizer": "lowercase",
        "type": "token"
      }
    }
  }
}

val searchStage = Aggregates.search(
  SearchOperator.compound()
    .must(
      Iterable(
        SearchOperator.in(fieldPath("genres"), List("Comedy")),
        SearchOperator.phrase(fieldPath("fullplot"), "new york"),
        SearchOperator.numberRange(fieldPath("year")).gtLt(1950, 2000),
        SearchOperator.wildcard("Love *", fieldPath("title")),
      ).asJava
    )
)
val projectStage = Aggregates.project(
  Projections.include("title", "year", "genres"))
val aggregatePipelineStages = Seq(searchStage, projectStage)
collection.aggregate(aggregatePipelineStages)
  .subscribe((doc: Document) => println(doc.toJson()),
    (e: Throwable) => println(s"There was an error: $e"))

{"_id": ..., "genres": ["Comedy", "Romance"], "title": "Love at First Bite", "year": 1979}
{"_id": ..., "genres": ["Comedy", "Drama"], "title": "Love Affair", "year": 1994}

MongoDB Server Manual

To learn more about the topics discussed in this guide, see the following pages in the MongoDB Server manual:

• To view a full list of expression operators, see Aggregation Operators.

• To learn about assembling an aggregation pipeline and to view examples, see Aggregation Pipeline.

• To learn more about creating pipeline stages, see Aggregation Stages.

• To learn more about explaining MongoDB operations, see Explain Output and Query Plans.

API Documentation

To learn more about the methods and types discussed in this guide, see the following API documentation:

• aggregate()

• Aggregates

• explain()