Sea of Tranquility

Don't typically do Sci-fi but Emily St. John Mandel is such a skilled storyteller I acquiesced. Here's the thing: this book is about pandemics. It's about time travel. In the opening she references the Europeans bringing smallpox to North America. So this gets me thinking, doesn't the nature of diseases make the theory of time travel impossible? Because surely people from the future will have developed strains of disease the immune systems of people in the past wouldn't be able to withstand? Effectively ‘ruining' the space time continuum or whatever?