In most cases, organized by director/creator. Within that director’s section, organized chronologically. TV series are mixed in (chronologically by the start of the series). Blu-rays and DVDs are mixed together. Criterion is mixed in.
That said, there are some movies that get lumped together even though they have different directors. The Star Wars trilogy goes together under “L”. The Studio Ghibli films go together under “M”. Sequels are put together even if there are unrelated one-offs in between, so the whole Star Wars trilogy goes first, then the Holiday Special is after.
It’s a really logical system… for me. The downside is that the rest of the family never knows where to put the movies back – “Dad, I’m trying to put back The Empire Strikes Back but I can’t find the ‘T’ section!” Which is fine because I’d rather do it anyway.