Sql-server – What are some real world applications that generate large numbers of records in a database every day and what are those metrics?

database-designperformancequery-performancesql serverstorage

I'm trying to get some perspective in today's day and age what is considered a large amount of data or large number of records generated in a day and what real world applications are doing that. I would guess large banks have to be generating many records every day in their systems, and I wonder just how many this actually is, hundreds of thousands, millions, hundreds of millions, etc?

How do they typically manage this much data every day from a data storage, query performance, and maintenance perspective?…how do they not run out of primary key values if they aren't using something like a GUID?

Best Answer

Trading. I was part of a DWH team for a larger trading outfit and we generated 110 million update rows per day on existing contracts. Requirement to keep them for years.

Now I dabble in the field, and my validation systems have peaks of around 500 million rows per day, but we only keep those for a week or so.

how do they not run out of primary key values if they aren't using something like a GUID?

Composite keys. Contract, date, update number. Stuff like that.