As a caveman, you don't need a lot of fancy deductions. Stick to the basics: rocks, sticks, and berries.
Keep your business and personal expenses separate. You don't want to get audited by the saber-toothed tiger tax authority.
As a caveman, you're not exactly swimming in wealth. Use your land to grow food, not to build a McMansion.
Barter for what you need. You'll save on taxes and get a good story to tell around the campfire.