I do a lot of brick. standard hw mix does fine. Need some soap on brick because of rougher texture holds dirt much more so than smoother siding. Bleach is bleach, doesn’t necessarily clean, helps eliminate mold and mildew, so you need soap. just not a clinging one. Reason I say you can’t always use one soap for everything. That’s just being lazy and cheap.
And yes, it does take a little longer to rinse brick, maybe couple of minutes per side, if that, because you’re not worried about leakers, corner boards, etc. Just be systematic about it.