The trick to the Spectral Librarian & Tome fight seems to be to make sure to keep their damage within a very narrow band of each other. This may not matter so much when they're green/yellow, but once they're below some (yet to be determined) threshold it is vital.