Refined grains are quickly digested into simple sugars and absorbed into your bloodstream, causing your blood sugar to spike and then crash. “This can leave you feeling tired and irritable,” notes Miller. Another (huge) downside to white bread and bagels? “Diets rich in refined grains can contribute to increased inflammation throughout the body,” says Miller.