The buildings at the back of this shot are the Ferry Point Estate along Man Cheong St - a street that, as you can see, was once right at the water's edge. The west Kowloon reclamation has put a stop to that and they are now marooned 300 metres inland.
These buildings make up the Ferry Point Estate. All eight of them are called Man...something and were built in stages throughout the 60's - the oldest ones were actually built circa 1964 and the last completed circa 1970. So they were fairly new when this film was made.
Judging from the angle of this shot the boat must have been somewhere off the eastern side of Tsim Sha Tsui - probably the area where the iconic Arch now stands (on solid ground!). The nearest is the Man King Building followed by Man Fai, Man Yiu and finally Man Cheong Buildings. These four are the ones that used to face the water and another four older ones on the right of these facing Ferry St.