Well, it probably does not feel kosher because you are letting go of what you know is an important control.
I think the bottom line on this is: if there is a problem, - say with someone being on OFACs list - you will probably be held responsible.
I think I would have an agreement, - I think I would reserve the right to inspect his files on the workers, and I think I would do that quarterly.
Also, because you are talking farms and the workers could be transient folks, I think I would ask for a list of the workers on a quarterly basis and run them through OFAC myself.
Just my thoughts.