Remember that these Minions are not as smart as they think they are at all, in fact they're quite immature in their intelligence department. Try again with offering him a different gem than a Diamond, and make sure you have one of every type of gem in your Inventory before you offer him any. If you don't have one of each to start with, remember too that you can trade down your gems in the same way as you trade them up. They always equal out on the trade-in value either way, whether it be from highest to lowest, or lowest to highest types of gems. ;)

Thanks, I didn't have agate and now he opened the door.