Distinct is a partially blocking component that remove duplicates from one flow. Its main advantages against the sort component provided in SSIS are:
- Memory usage: Distinct does not cache the whole flow (as Sort does) but retains in memory only the distincts, consuming less memory then Sort
- Distinct is partially blocking where Sort is fully blocking
- Distinct is freeware, you can easily download sources and adapt it to your needs
A full description of the component and full sources can be downloaded at www.sqlbi.eu.
If you would like to receive an email when updates are made to this post, please register here
Subscribe to this post's comments using
Alberto Ferrari is a Business Intelligence consultant.
He his interests lie in two main areas: BI development lifecycle methodologies and performance tuning of ETL and SQL code. His main activities are with SSIS and SSAS for the banking, manufacturing and statistical sectors.
He is also a speaker in international conferences like European PASS Conference and PASS Summit.