The White Ship

The story doesn't really go anywhere, and it's not quite a horror story. It's apparently one of his earlier works and tells the tale a lighthouse keeper who went on a wistful journey and visited far shores and fantastic cities. There appears to be a moral story here though, as he ends with his hopes dashed and lost. With Lovecraft's penchant for flowery prose, you can probably imagine where the bulk of the text goes to.