When you arrive in Iceland very early in the morning or very late at night there are several things that make driving straight away a bad idea.

It is very likely that you have had little or no sleep which means that the risk of falling asleep at the wheel is very high.
The time difference can cause jet lag which again causes fatigue and other sleep related symptoms.

Conditions in Iceland can be very different from what you are used to which can be challenging to face, expecially when tired and in some cases in darkness.
The trip to your final destination can take several hours and pitstops can be few and far between. It is very unwise to start such a trip when sleepy.

What you could do instead is check in to a hotel near the airport at a special Nap&Go rate and get a few hours sleep.
It costs less than you might think and it will make the whole day more enjoyable.

You will be much safer and more alert with less risk of falling asleep at the wheel.
You will enjoy your travel much more, being well rested and in better conditions to drive.
You will enjoy the scenery more in bright daylight.