Testing if High Dose Radiation Only to the Sites of Brain Cancer Compared to Whole Brain Radiation That Avoids the Hippocampus is Better at Preventing Loss of Memory and Thinking Ability

Inclusion Criteria: * Pathologically (histologically or cytologically) proven diagnosis of small cell lung cancer within 5 years of registration. If the original histologic proof of malignancy is greater than 5 years, then pathological (i.e., more recent) confirmation is required (e.g., from a systemic or brain metastasis); * Patients with de novo or recurrent small cell lung cancer are permitted. * Brain metastases =\< 4 cm in largest diameter and outside a 5-mm margin around either hippocampus must be visible on contrast-enhanced magnetic resonance imaging (MRI) performed =\< 21 days prior to study entry. * The total tumor volume must be 30 cm\^3 or less. Lesion volume will be approximated by measuring the lesion's three perpendicular diameters on contrast enhanced, T1-weighted MRI and the product of those diameters will be divided by 2 to estimate the lesion volume (e.g. xyz/2). Alternatively, direct volumetric measurements via slice by slice contouring on a treatment planning software package can be used to calculate the total tumor volume. * Brain metastases can be diagnosed synchronous to the initial diagnosis of small cell lung cancer or metachronous to the initial diagnosis and management of small cell lung cancer. * Brain metastases must be diagnosed on MRI, which will include the following elements: * REQUIRED MRI ELEMENTS * Post gadolinium contrast-enhanced T1-weighted three-dimensional (3D) spoiled gradient (SPGR). Acceptable 3D SPGR sequences include magnetization prepared 3D gradient recalled echo (GRE) rapid gradient echo (MP-RAGE), turbo field echo (TFE) MRI, BRAVO (Brain Volume Imaging) or 3D Fast FE (field echo). The T1-weighted 3D scan should use the smallest possible axial slice thickness, not to exceed 1.5 mm. * Pre-contrast T1 weighted imaging (3D imaging sequence strongly encouraged). * A minimum of one axial T2 FLAIR (preferred) or T2 sequence is required. This can be acquired as a two dimensional (2D) or 3D image. If 2D, the images should be obtained in the axial plane. * ADDITIONAL RECOMMENDATIONS * Recommendation is that an axial T2 FLAIR (preferred) sequence be performed instead of a T2 sequence. * Recommendation is that that pre-contrast 3D T1 be performed with the same parameters as the post-contrast 3D T1. * Recommendation is that imaging be performed on a 3 Tesla (3T) MRI. * Recommendation is that the study participants be scanned on the same MRI instrument at each time point. * Recommendation is that if additional sequences are obtained, these should meet the criteria outlined in Kaufmann et al., 2020. * If additional sequences are obtained, total imaging time should not exceed 60 minutes. * If additional metastases not known at the time of registration/randomization or seen in the MRI used for eligibility are subsequently found on the radiation therapy (RT) planning MRI such that the total intacranial volume exceeds 30 cm\^3, the patient is still considered eligible. * History/physical examination * Age \>= 18 * Karnofsky performance status of \>= 70 * Creatinine clearance \>= 30 ml/min * Following the diagnosis of brain metastases, patients can initiate and treat with systemic (chemotherapy and/or immunotherapy) before enrollment only if their brain metastases are asymptomatic and not located in eloquent locations (e.g., brainstem, pre-/post-central gyrus, visual cortex). However, within 21 days prior to enrollment, brain MRI must be repeated to confirm eligibility. * Patients with symptomatic brain metastases and/or brain metastases in eloquent locations (e.g., brainstem, pre-/post central gyrus, visual cortex) are eligible for enrollment on the trial; however, the specific treatment approach of starting with systemic therapy alone and delaying brain radiation is not recommended for these patients. * Concurrent immunotherapy with brain radiation (SRS or HA-WBRT) is permitted. * Negative urine or serum pregnancy test (in women of childbearing potential) within 14 days prior to registration. Women of childbearing potential and men who are sexually active must use contraception while on study. * Patients may have had prior intracranial surgical resection. * Because neurocognitive testing is the primary goal of this study, patients must be proficient in English or French Canadian. * The patient must provide study-specific informed consent prior to study entry. * Patients with impaired decision-making capacity are not permitted on study. * ELIGIBILITY CRITERIA PRIOR TO STEP 2 REGISTRATION * The following baseline neurocognitive tests must be completed within 21 days prior to Step 2 registration: HVLT-R, TMT, and COWA. The neurocognitive test will be uploaded into RAVE for evaluation by Dr. Wefel. Once the upload is complete, within 3 business days a notification will be sent via email to the RA to proceed to Step 2. * NOTE: Completed baseline neurocognitive tests can be uploaded at the time of Step 1 registration. * PRIOR TO STEP 2 REGISTRATION: The following baseline neurocognitive tests must be completed within 21 days prior to Step 2 registration: HVLT-R, TMT, and COWA. The neurocognitive tests will be uploaded into RAVE for evaluation by Dr. Wefel. Once the upload is complete, within 3 business days a notification will be sent via email to the RA to proceed to Step 2. NOTE: Completed baseline neurocognitive tests can be uploaded at the time of Step 1 registration. Exclusion Criteria: * Planned infusion of cytotoxic chemotherapy on the same day as SRS or HA-WBRT treatment. Patients may have had prior chemotherapy. Concurrent immunotherapy is permitted. * For patients receiving fractionated SRS on an every-other-day basis, planned infusion of cytotoxic chemotherapy is not permitted between SRS treatments. * Brainstem metastasis \> 10 cm\^3 * Prior allergic reaction to memantine. * Patients with definitive leptomeningeal metastases. * Known history of demyelinating disease such as multiple sclerosis. * Contraindication to MR imaging such as implanted metal devices that are MRI-incompatible, allergy to MRI contrast that cannot be adequately addressed with pre-contrast medications, or foreign bodies that preclude MRI imaging. (Questions regarding MRI compatibility of implanted objects should be reviewed with the Radiology Department performing the MRI). * Current use of (other N-methyl-D-aspartate \[NMDA\] antagonists) amantadine, ketamine, or dextromethorphan. * Radiographic evidence of hydrocephalus or other architectural change of the ventricular system resulting in significant anatomic distortion of the hippocampus, including placement of external ventricular drain or ventriculoperitoneal shunt. * Mild cases of hydrocephalus not resulting in significant anatomic distortion of the hippocampus are permitted. * Prior radiotherapy to the brain, including SRS, WBRT, or prophylactic cranial irradiation (PCI).