We are a $320 million bank. I'm the Risk Manager and Compliance Officer. Our Operational Risk Working Group, which has representation from across the bank, completes the annual information security risk assessment. I prepare the final documentation, but the individual areas are the source of information/content experts. We outsource internal audit, and many of the items in the risk assessment are audited by our outsourced internal audit function.
I don't see any issue with line of business managers completing the risk assessment - in fact, I think that is the way to go. They are the ultimate risk owners, so they need to make sure they are understanding the risk in their area, and taking appropriate steps to mitigate what the institution considers an unacceptable level of risk. My guess is that your IT officer is like a lot of IT people who absolutely loathe the documentation process. In our institution, we've recognized that even if we force IT to document, it's not their strength and they don't do a great job at it - which is where I come in. I get all of the information/content out of them, and am ultimately the one who writes it down - but it's not my risk assessment - it's still theirs.
Another point, however, is that the IT officer is not the one solely responsible for the information security risk assessment. It goes far beyond technology and systems. All line of business managers in the institution should be involved in the risk assessment and risk mitigation process.