We've got this one down to a science. Simply tell them you're not available.
Example: "Sorry, I'm not available for the meeting. My cat has a hairball."
By consistently employing this technique, you'll gain an air of mystery and become the office whisperer.
Don't worry, it's not just you - it's science.