On Monday, White House Chief of Staff John Kelly characterized the removal of Confederate statues as a “dangerous” erasure of history.

Kelly was speaking to Fox News’ Laura Ingraham at the launch of her new show, “The Ingraham Angle,” when Ingraham mentioned the removal of plaques honoring President George Washington and Gen. Robert E. Lee in a Virginia church. The church characterized the move as being necessary for inclusion.

“I think we make a mistake, though, and as a society and certainly as, as individuals, when we take what is today accepted as right and wrong and go back 100, 200, 300 years or more and say: ‘What Christopher Columbus did was wrong,’” Kelly said. “You know, 500 years later, it’s inconceivable to me that you would take what we think now and apply it back then.”

He then defended Lee, the leader of the Confederate army.

“I would tell you that Robert E. Lee was an honorable man,” Kelly said. “He was a man that gave up his country to fight for his state, which, 150 years ago, was more important than country. It was always loyalty to state first back in those days. Now it’s different today. But the lack of an ability to compromise led to the Civil War. And men and women of good faith on both sides made their stand where their conscience had to make their stand.”