Fenyx, who manages the Codex of Ultima Wisdom wiki, has released a utility on Github for viewing maps of the dungeons in Ultima 4:

Tool made for the The Codex of Ultima Wisdom for displaying dungeons

Files from Ultima games are not included in this repository. You will need to place *.DNG files, CHARSET.EGA and WORLD.MAP from PC version of Ultima 4 in the Maps directory.

If you for some reason don’t have an on-hand copy of Ultima 4, you can grab one for free from GOG. Fenyx’s tool, meanwhile, can be downloaded from its Github page.