I think AVCHD discs are supposed to be in UDF 2.5, but I'm not positive.
You definitely don't want HFS+, that's Mac-only. I'd go with UDF+ISO9660 myself, but ISO9660+Joliet might work too...
Just looked into it - you definitely need to burn in UDF 2.50 for it to work on a PS3. UDF+ISO9660 won't work.