Right click Nibbles to kick him a few times. He will then become aggro.
In this room there are three doors: south, east, and west. A short time after the fight starts, you will see a red message that will tell you that dark energy is in the direction of two doors; for example, if SOUTH and EAST are called, exit the room through the WEST door and take a breath of fresh air. After several seconds Nibbles will yell "BACK! BACK!" and you will automatically be ported back into the room to resume fight. If you attempt to exit through the incorrect door after a warning message, an undead critter will spawn and attack you until it is dead or the encounter ends.
This will also complete the "Flight of the Nibbler" Achievement and allow you to buy Nibbler cosmetic pets and house items from the vendor at the Rum Cellar docks.