Now, there is still a side entrance at this point of the hotel but much has changed since 1965. The doorway now leads into the so-called Peninsula Arcade - basically a shopping mall containing the usual high end shops. In 1965 though, when these episodes were filmed this door entered into the hotel proper. As you can see from the brass plate in the current Streetview below, this is also now the entrance to Gaddi's, which is now situated up the stairs (it used to be on the ground floor). You should also be able to see that along with being turned into a mall, the exterior of the ground floor has also been extensively remodeled so that the shops have external window displays.
A second after the man in the hat follows Culp and Cosby, the camera pans across Nathan Road to the opposite side and we get a quick glimpse of another hotel that has since departed: The Ambassador.
The Ambassador was quite striking courtesy of a large mural on the Nathan Road-facing wall. Gwulo.com has a good closeup here: http://gwulo.com/node/14902. Although I'm unsure when it disappeared - probably when Oterprise Square replaced it (whenever that was, it's build date seems to be eluding me at the moment) - it was built in 1960 and so was only a few years old as we see it above.
Eagle-eyed readers may also have noticed the rather large space standing on the south side of Middle Road (on the other side of the bus in the pic above). This of course is now the Sheraton Hotel, but in 1965 was just a large open car park. The hotel didn't appear until 1973.