Mongo Playground

Configuration

[{"key":1},{"key":2},{"key":{"value":1}},{"key":{"value":3}}]

Query

db.collection.aggregate([{"$addFields":{"key2":{"$cond":{"if":{$and:[{"$eq":[{"$type":"$key"},"object"]}]},"then":"$key.value","else":"$key"}}}},{"$group":{"_id":"$key2","data":{$push:"$$ROOT"}}}])

Result