The Camera Calibrator can handle some loss of targets and still provide a well calibrated camera.
The Single Sheet Calibration grid has 100 points and if 80 or more are marked per image then that is sufficient. If one or two images do not get used because not all four control points were found then that will generally be OK as well, but it is recommended that all 4 control points are clearly visible in each photo.
If too many photographs are dropped due to problems you will want to investigate. Marking problems may be due to: lack of sharp focus on the image, too much of the grid cropped off, the grid shot at too shallow an angle, or undue sharpening done by a digital camera.
If targets in the background are missed (i.e. the smallest ones), it may be that they are too small to be recognized, based on current preference settings. There is a threshold setting on the Preferences dialog that can be adjusted so that smaller targets are detected. See the “Ratio of smallest target in image to image size” setting in Preferences - Processing and Cameras.
Resolve the problem, reshoot the photos and try again.