The only downside would be if you forgot to swap out the 6mm bushing when you are sizing 6.5mm brass.
Doing the same for 6.5 and 22CM, also using 6CM Forster ultra micrometer die for my 22CM, as you said, works like a charm.Using a Redding 6.5 Creedmoor "S" die for 6.5mm, 6mm and 22 Creedmoors. Works like a charm.