The F-4's got guns, either built in or attached under the nose cone, on later versions, including the Navy versions.
The primary reason that it didn't hit the streets with a gun initially was that the military thought the missiles would be effective, but they were fragile after rough carrier landings and had to be diagnosed and serviced between flights. However, the pilots demanded that the missiles remain on the wings while the planes refueled, to allow a faster turn around for the planes to go back in the air. Thus, when the planes had several carrier landings, and the missiles were not tended to downstairs, they often mis-fired when they finally got a chance to attack a MIG. They wither fell off and did not ignite, or they streaked to target and did not explode, or they missed the target, or they went past the target and exploded harmlessly.
In my time as a co-op at Mac I reviewed hundreds of debriefs of F-4 missions, and it was massive evidence that a gun was necessary on the F-4. It took a while for the upgrades, and the external gun was mounted on planes until the internal gun was in the production planes. For some reason the Navy stayed with the external gun (I may be missing the possibility that the Navy also got the internal gun on some of it's models.)
Also, the F-4, while heavier than the Mig 21 or Mig 19 or Mig 17, it had more thrust, and a strategy was devised for dog fighting that used it's strengths and minimized it's weaknesses. (Similar to the WWII F4F against the Zero). The pilots learned to avoid level turns against the more maneuverable MIG's, and go vertical instead. It could out climb the MIG's. So The F-4 attacked with a firing pass, and if it missed the MIG it went up or down and disengaged briefly until it was in a favorable position again. PLUS . . . most of the engagements involved a numerical advantage for the Americans, usually 4 to 1 or 4 to 2, so they ganged up on the MIG's. Most of the engagements involved a long chase of the escaping MIG as it headed north to safety. (There was some line in N Vietnam that the F-4's were not allowed to cross, something about Russians being harmed north of the line.)
One of the biggest lessons that F-4 pilots learned was the use of a 'trailer' by the MIG pilots. The F-4 would encounter a lone MIG and turn in for the kill, while a second MIG was a mile or two behind in the trailer position and get behind the attacking the turning F-4's. The Americans learned to attack the trailer as the primary target and then head back for the lead plane, which was often piloted by a less capable pilot. Numerous examples recounted a chase to the north of the escaping MIG with a young pilot who could not even make coordinated turns in the plane as he tried to escape. A bunch of the MIG kills resulted in other F-4's being vectored in on an escaping MIG before it could get away. The pilots liked to make it sound like they shot down the Red Baron, but many times it was a junior level throw away pilot in the MIG. Apparently the N Vietnamese did not produce the quantity of expert pilots that they needed, so less qualified pilots had to learn on the job against a highly trained American force. I've heard similar stories in engagements in the Middle East. The Americans and Israelis have a supremely trained pilot group, and the opponents have some good and some less good in their pilot core. I assume that the Russians are much more qualified than their proxies, anywhere in the world that they have proxies using Russian planes. (Similar in Korea, where it is well known that they needed Russian pilots in the MIG's to fight on an equal footing.)
Beyond that, the F-4 was very rugged after receiving battle damage, some compensation for the excess weight it had to haul. Two engines is a major plus also, because lot's of times it is engine damage that causes the loss of a plane. One problem though, is that the engines were packed right next to each other, so the shrapnel from one exploding engine sometimes took out the other engine too. That explains why the F-15 had it's engines spaced several feet apart, with metal in between, so that the loss of one engine did not cause the loss of the second engine. Plus there are other advantages to a plane being flatter and wider, but not too much, because when the weight moves outboard, the turning ability is slower. (That is why the P-38 did not turn as well as single engine planes with all the weight along the center line).
The military wants EVERYTHING in a plane, but long story short, the plane is primarily a winged weapons platform, a delivery system. Many of the problems they face, they solve with smarter and more reliable weapons systems, and they want a pounds for dollars ratio for the weapons systems. They would rather not get close enough to dogfight anymore, but instead smoke the opponent at a distance. When they have to dogfight, they like numbers in their favor. And when the plane is good enough, like the F-15, they will gladly wail into the enemy and smoke him. And they like supporting aircraft in the skies over the battle field to support with electronics and communication and radar, etc, that no enemy so far has matched. And what they REALLY want is a pilot-less dog fighter that can take 20 G's in turns, far more than any manned plane could possibly approach.
And all I get to do is Combat Flight Simulator from Microsoft. Never wet my pants flying that way.