YTD or annualised returns - which is more accurate to measure my portfolio performance? In what kind of situation do I use each of the return as measurement?

Annualised returns to track performance over the years, and Year To Date to track this year's performance against a benchmark.

You can use both.

Annualised returns is important to see if your money has actually been growing consistently. YTD just to see if I'm outperforming or underperforming against economic sentiments this year alone. YTD can change very quickly, Annualised Performance especially if your portfolio has been around for many many years, won't change too quickly.