Efficiently remove duplicate lines from text or find which lines are duplicated. Includes options for trimming whitespace and removing empty lines before processing.