In my (huge) Sitecore solution we experienced the following error:
ERROR PostSessionEndPipeline failed.
Exception: MongoDB.Driver.MongoQueryException
Message: QueryFailure flag was Runner error: Overflow sort stage buffered data usage of 33570904 bytes exceeds internal limit of 33554432 bytes (response was { “$err” : “Runner error: Overflow sort stage buffered data usage of 33570904 bytes exceeds internal limit of 33554432 bytes”, “code” : 17144 }).
Source: MongoDB.Driver
at Sitecore.Analytics.Pipelines.CommitSession.SubmitSession.Process(CommitSessionPipelineArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Analytics.Pipelines.PostSessionEnd.CommitSession.Process(PostSessionEndArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Web.Application.RaiseSessionEndEvent(HttpApplication context)
Followed by the following message:
QueryFailure flag was Runner error: Overflow sort stage buffered data usage of xxxxxxxx bytes exceeds internal limit of 33554432 bytes (response was { “$err” : “Runner error: Overflow sort stage buffered data usage of xxxxxxxx bytes exceeds internal limit of 33554432 bytes”, “code” : 17144 }).
The error is caused by an internal 32MB memory size restriction in MongoDB.
The problem can be solved by adding an index to the Interactions collection. This will allow Sitecore to stream interactions sorted rather than attempting to load them all into memory and do the sorting in memory.
Connect to the analytics index using RoboMongo or MongoVUE and run the following query:
db.Interactions.createIndex(
{
"StartDateTime" : 1,
"_id" : 1
}
)
A WORD OF CAUTION:
According to MongoDB, adding indexes to a live environment does not have any side effects. But running the above query on my 7.5 million entries in the Interactions collection caused my entire Sitecore to fail (total catastrophic failure) for 5-10 minutes until the index was built.
Thank you to Sitecore Support and Joshua Wheelock for helping with the issue.
MORE TO READ:
- Overflow sort stage buffered data usage exceeds internal limit by stackoverflow.
