There is going to be a problem with perception so this could eventually cause you to lose your job. Or other employees could become resentful. I have always been early, so I have to admit I do not have a lot of patience with someone who is always late. I have a few colleagues that are always doing their copying at the last minute, and are then always late for their classes. They are also always late for meetings. They view themselves as laid back; I view them as procrastinators and as being rude. I also see a tendency in certain students to always be late. Some of my colleagues tell the students that they can not come in once the door shuts. Our classroom doors automatically lock when closed. While I have threatened to do that, I never have. If I do have a student that is consistently late, I will no longer give them attendance/participation points on the days they are late.
You have been given some good suggestions.