The thing about Goblet of Fire is that there's an awful lot of expository information in the book that provides a primer on the previous 3 books; that stuff can all be dropped for the film, although the current rumour is that there will be no scenes involving the Dursleys in the Goblet of Fire movie.
As for Prisoner of Azkaban, I think it's going to work well. Alfonso Cuaron has already shown his ability to work with kids, and has a unique visual style that suits the darker tone of the 3rd book.