I did a program called JET. All you need is a bachelor's degree (doesn't matter what field), fluency in English, a pulse, and to not be a total fuck-up. Even then, the last two are debatable. That's it. You don't need any teaching experience (I had NONE prior to coming) or any Japanese language abilities.
JET actually discourages the use of Japanese in some ways (after all, they're not bringing you to Japan to speak Japanese with Japanese people), but I personally think it's extremely useful, inside of the classroom and especially outside. Most of the JET's who come don't actually have that much Japanese ability (most of my friends couldn't speak Japanese when they got here, some of them still can't). If you really want to learn, well, there's no better way than total immersion. There'll be plenty of opportunities to do so, and it would be good for your students to see you learning their language as they learn yours.
If you are really interested in JET, you can inquire about it at your university's Career Center, Japanese Department, or go to the JET Program Website.
There are other programs besides JET, which place people in conversation schools like Nova and ECC. I don't know too much about them, so I can't be of much help there. From everything I've heard though, they're not quite as good as JET. I believe the requirements are the same though - you just need a bachelor's degree and to be fluent in English.
