Devil's Peak - the hill which sits above Lei Yue Mun - has always been militarily significant and as such has had some sort of outpost on it ever since people were living here. Even before the British came along and built a concrete fortress up there, it was used by pirates as a lookout post simply because it offered a great eastern view out to see and incoming ships could be seen many miles away.
This latter fact doesn't seem top have escaped Jackie Chan when he used it as the pirates lookout in Project A. Here is a grab of the pirates looking out of the windows of the old fort.
Anyone who has hiked up here should be able to recognise the slanted viewing ports that line the fort wall on the eastern side of Devil's Peak. Here is a shot I took a few years ago (now you know how long I have been putting this stuff off for) albeit taken from the other end looking back towards where the camera was positioned.