How come changing the indexes didn't change the order of the list items?

When the foreach function was used to iterate through the array, shouldn't it start at the lowest index in the array and then go on to the next one? So when the indexes were changed how come the order of the list items stayed the same even though the last list item has the second lowest index?

Here is the code for an example:

Creating the array and setting the indexes:

<?php 
  $catalog = array();
  $catalog[101] = "Design Patterns";
  $catalog[201] = "Forest Gump";
  $catalog[301] = "Beethoven";
  $catalog[102] = "Clean Code";
?>

Then iterating through the array to add it to an unordered list:

<?php
  foreach($catalog as $item){
    echo "<li>" . $item . "</li>";
  }
?>

Shouldn't the outcome be:

<ul>
  <li>Design Patterns</li>
  <li>Clean Code</li>
  <li>Forest Gump</li>
  <li>Beethoven</li>
</ul>

But instead it comes out:

<ul>
  <li>Design Patterns</li>
  <li>Beethoven</li>
  <li>Forest Gump</li>
  <li>Clean Code</li>
</ul>

So why is this? Does foreach not start at the lowest index but instead goes through the order in which the items were added to the array? If so how does it know to do that and why doesn't it start at the lowest index?

Thanks.