Stereotactic Body Radiation Therapy or Intensity-Modulated Radiation Therapy in Treating Patients With Stage IIA-B Prostate Cancer

Inclusion Criteria: * Previously untreated (no local therapy such as surgery, radiation cryotherapy, high-intensity focused ultrasound \[HIFU\], etc.) localized adenocarcinoma of the prostate with the following clinical findings: * Clinical stage by digital rectal exam of either T1c or T2a/b (limited to one side of the gland); (American Joint Committee on Cancer \[AJCC\], version 7) or cT1a-c or 2a or 2b * Stages T1a-T1b are eligible if patient underwent transurethral prostatic resection (TURP) * The patient must meet one of the following 3 criteria: 1) Gleason score must be Gleason 7(3+4) with a PSA \< 20 ng/mL, or 2) Gleason 6 (3+3) with a PSA \> 10 ng/mL and \< 20 ng/mL which is considered intermediate risk and eligible for the study (AJCC, version 7), or 3) Group Grade 1 with a PSA \> 10 ng/mL and \< 20 ng/mL or 2 with a PSA \< 20 ng/mL * If patient is receiving a 5-alpha reductase inhibitor at the time of enrollment, the baseline PSA value may be double the initial value and the medication should be discontinued but a washout period is not required; to be eligible, a PSA drawn while still on the medicine must be: * \< 10 ng/mL if Gleason 7(3+4) (note this patient would be on stratification level 1 if PSA \< 5 ng/mL and stratification level 2 if less than 10 ng/mL) * \> 5 ng/mL and less than 10 ng/mL for Gleason 6(3+3) (note this patient would be on stratification level 3) * The prostate volume must be \< 70 cc as reported at time of biopsy or by separate measure with ultrasound or other imaging modalities including magnetic resonance imaging (MRI) or computed tomography (CT) scan * Patients in active surveillance who elect to be treated are eligible if they meet protocol requirements * Age \>= 18 * History and physical including a digital rectal exam 60 days prior to registration * Eastern Cooperative Oncology Group (ECOG) performance status 0-1 60 days prior to registration * MRI of the prostate and pelvis (per institutional standard of care \[SOC\] - should be compliant with PIRADSv2.1 guidelines) within 1 year prior to registration * Bone scan as clinically indicated within 120 days prior to registration * Charlson modified co-morbidity score =\< 4 for patients under 60 and =\< 5 for patients 60 and over 21 days prior to registration * International prostate symptom score (IPSS) of \< 15 21 days prior to registration * The patient must provide study-specific informed consent prior to study entry * Willingness and ability to complete the Expanded Prostate Cancer Index Composite (EPIC-26) questionnaire * Completion of all items of the EPIC-26 which will be data entered at registration 60 days prior to registration * Only English, Spanish, and French-speaking patients are eligible to participate as these are the only languages EPIC-26 has been validated in Exclusion Criteria: * Definitive clinical or radiologic evidence of metastatic disease; no nodal involvement or evidence of metastatic disease allowed as defined by screening of the pelvis * Definitive T3 disease on MRI * Prior or current invasive malignancy with current evidence of active disease within the past 2 years * Exceptions: Non-melanomatous skin cancer, carcinoma in situ of the male breast, penis, oral cavity, or stage Ta of the bladder, or stage I completely resected melanoma * Prior systemic chemotherapy for the study cancer; note that prior chemotherapy for a different cancer is allowable; must be off treatment for at least 3 years * Prior radiotherapy to the region of the study cancer that would result in overlap of radiation therapy fields * The use of hormonal therapy is not allowed; if the patient is on a 5-alpha reductase inhibitor, then they should be stopped prior to treatment once enrolled onto the study; no washout period is required for this study to participate * Severe, active co-morbidity defined as follows: * Human immunodeficiency virus (HIV) positive with CD4 count \< 200 cells/microliter; Note that patients who are HIV positive are eligible, provided they are under treatment with highly active antiretroviral therapy (HAART) and have a CD4 count \>= 200 cells/microliter within 30 days prior to registration; Note also that HIV testing is not required for eligibility for this protocol; this exclusion criterion is necessary because the treatments involved in this protocol may be significantly immunosuppressive * Hepatic insufficiency resulting in clinical jaundice and/or coagulation defects; note, however, that laboratory tests for liver function and coagulation parameters are not required for entry into this protocol; (patients on Coumadin or other blood thinning agents are eligible for this study) * Contraindication to MRI * Cardiac pacemaker or defibrillator * Surgically implanted electrical devices such as spinal stimulation devices or intracranial stimulation devices, cochlear implants, the presence of metallic foreign bodies in the orbits, and incompatible old mechanical heart valves and aneurysm clips