Is it allowed to walk into a church?

Of course it is allowed. If the church is open for prayer, then just walk in and sit wherever. No one will mind.