I understand the importance of challenging my own prejudices, but I can't really take responsibility for the larger, macro-level institutional racism that you are describing. It's not my fault.

No, institutional racism is not any one person's fault, but it is the responsibility of everyone participating in that system. It is especially the responsibility of those benefiting from institutional racism. It is the individual, psychological level of prejudice that allows us to justify the unfairness and destructiveness of institutionalized racism. Individual-level prejudice allows us to justify the outcome of unfair systems ("those people deserve what they get...they're lazy and have to work harder," or even "I don't really care what happens to them"). Challenging our own prejudices at the psychological level then means that we also have to challenge larger structures of racism in society, as well as the prejudices of other individuals which support those structures. This is why people used to say 20 years ago, "If you're not part of the solution, you're part of the problem."