He cautions us to look at what the Bible throughout has required.
His test for whether someone is religious includes care for orphans and widows in distress and keeping oneself unstained from the world. Some of us think that one of those is more important than the other. James is holding out for both--as well as refraining from anger.
Being religious might be hard for some of us.