Testing the Addition of the Anti-cancer Drug, Cabozantinib, to the Usual Immunotherapy Treatment, Avelumab, in Patients With Metastatic Urothelial Cancer, MAIN-CAV Study

Inclusion Criteria: * Histologically or cytologically-confirmed diagnosis of advanced or metastatic urothelial cancer of the renal pelvis, ureter, bladder, or urethra (transitional cell and mixed transitional/non-transitional cell histologies except for small-cell histology), including N3 only disease prior to start of first-line platinum-based chemotherapy * Prior first-line treatment must have consisted of 4-6 cycles of 1st-line therapy (platinum-based chemotherapy; gemcitabine-cisplatin, gemcitabine-carboplatin, methotrexate, vinblastine, doxorubicin and cisplatin \[MVAC\] or dose-dense \[dd\]MVAC) * No more than 1 line of prior chemotherapy for metastatic or locally advanced disease (neoadjuvant or adjuvant chemotherapy will be allowed if given 12 or more months prior to registration) * Tumor objective response of CR, PR, or SD upon completion of first line platinum-based chemotherapy by treating physician's assessment * The last dose of first-line chemotherapy must have been received no less than 3 weeks, and no more than 10 weeks, prior to randomization in the present study * No prior immunotherapy with IL-2, IFN-alpha, or an anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CD137, or CTLA-4 antibody (including ipilimumab), or any other antibody or drug specifically targeting T-cell co-stimulation or immune checkpoint pathways * Age \>= 18 years * Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1 * Not pregnant and not nursing, because this study involves an agent that has known genotoxic, mutagenic and teratogenic effects * Women of childbearing potential must have a negative pregnancy test =\< 14 days prior to registration. * Women of childbearing potential include women who have experienced menarche and who have not undergone successful surgical sterilization (hysterectomy, bilateral tubal ligation, or bilateral oophorectomy) or are not postmenopausal. Post menopause is defined as amenorrhea \>= 12 consecutive months. Note: women who have been amenorrheic for 12 or more months are still considered to be of childbearing potential if the amenorrhea is possibly due to prior chemotherapy, antiestrogens, ovarian suppression or any other reversible reason * No use of immunosuppressive medication within 7 days prior to randomization except: * Intranasal, inhaled, topical steroids, or local steroid injections (e.g., intra-articular injection); * Systemic corticosteroids at physiologic doses =\< 10 mg/day of prednisone or equivalent; * Steroids as premedication for hypersensitivity reactions (e.g., computed tomography \[CT\] scan premedication) * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * Patients with diabetes type I, vitiligo, psoriasis, or hypo or hyperthyroid disease not requiring immunosuppressive treatment are eligible * None of the following: * Active autoimmune disease that might deteriorate when receiving the anti PD-L1 agent, avelumab. * No known symptomatic central nervous system (CNS) metastases. Patients with previously diagnosed CNS metastases are eligible if they have completed their treatment and have recovered from the acute effects of radiation therapy or surgery prior to randomization, have discontinued corticosteroid treatment for at least 2 weeks, and are neurologically stable. Baseline brain imaging with contrast-enhanced CT or MRI scans for subjects with known brain metastases is required to confirm eligibility. * No major surgery within 4 weeks prior to randomization. Subjects must have complete wound healing from surgery before randomization. Subjects with clinically relevant ongoing complications from prior surgery are not eligible. * No palliative radiotherapy within 48 hours prior to patient randomization. * No hemoptysis of ≥ 0.5 teaspoon (2.5 mL) of red blood, clinically significant hematuria, hematemesis, coagulopathy, or other history of significant bleeding (eg. Pulmonary hemorrhage) within 3 months before randomization. * No known cavitating pulmonary lesion(s) or known endobronchial disease manifestation. * No administration of a live, attenuated vaccine within 30 days prior to randomization. The use of inactivated (killed) vaccines for the prevention of infectious disease is permitted. The use of COVID-19 vaccines is permitted. * No uncontrolled, significant intercurrent or recent illness including, but not limited to, the following conditions: * Cardiovascular disorders including: * Congestive heart failure (CHF): New York Heart Association (NYHA) class III (moderate) or class IV (severe) at the time of screening. * Concurrent uncontrolled hypertension defined as sustained blood pressure (BP) \> 150 mm Hg systolic, or \> 90 mm Hg diastolic despite optimal antihypertensive treatment. * The patient has a known history of corrected QT interval calculated by the Fridericia formula (QTcF) \> 500 ms and confirmed by electrocardiogram (ECG) within 28 days before randomization. Note: if initial QTcF is found to be \> 500 ms, two additional electrocardiograms (EKGs) separated by at least 3 minutes should be performed. If the average of these three consecutive results for QTcF is ≤ 500 ms, the subject meets eligibility in this regard. * Any history of congenital long QT syndrome. * Stroke, transient ischemic attack (TIA), myocardial infarction, or other symptomatic ischemic event or thromboembolic event (eg, deep venous thrombosis, pulmonary embolism \[DVT/PE\]) within 6 months before randomization. Subjects with a diagnosis of incidental, subsegmental PE or DVT within 6 months are allowed if asymptomatic and stable at screening and treated with low molecular weight heparin (LMWH) or the direct factor Xa inhibitors rivaroxaban, edoxaban, or apixaban for at least 1 week before randomization. Non-symptomatic white matter disease in the brain is acceptable. * No significant gastrointestinal disorders, particularly those associated with a high risk of perforation or fistula formation including unresolved active peptic ulcer disease, cholecystitis, diverticultis, symptomatic cholangitis or appendicitis, or malabsorption syndrome within 28 days of randomization. * No other clinically significant disorders such as: * Any active infection requiring systemic treatment within 14 days before randomization. Subjects receiving oral (including prophylactic) antibiotics with no symptoms of infection at randomization are eligible. * Serious non-healing wound/ulcer/bone fracture within 28 days before randomization * History of organ or allogeneic stem cell transplant * No persisting toxicity related to prior therapy grade \> 2 constituting a safety risk based on the investigator's judgment. * No diagnosis of any other malignancy within 3 years prior to randomization, except for locally curable cancers that have been adequately treated such as basal cell or squamous cell skin cancer, or carcinoma in situ of the breast or of the cervix, Gleason \< 7 prostate cancer on surveillance without any plans for treatment intervention (eg, surgery, radiation, or castration), or prostate cancer that has been adequately treated with prostatectomy or radiotherapy and currently with no evidence of disease or symptoms and no indication for treatment. * No concomitant anticoagulation with coumarin agents (e.g., warfarin), direct thrombin inhibitors (e.g., dabigatran), direct factor Xa inhibitor betrixaban, or platelet inhibitors (e.g., clopidogrel). * Allowed anticoagulants are the following: * Prophylactic use of low-dose aspirin for cardio-protection (per local applicable guidelines) and low-dose low molecular weight heparins (LMWH).Therapeutic doses of LMWH or anticoagulation with direct factor Xa inhibitors rivaroxaban, edoxaban, or apixaban in subjects without known brain metastases who are on a stable dose of the anticoagulant for at least 1 week before first dose of study treatment without clinically significant hemorrhagic complications from the anticoagulation regimen or the tumor * Absolute neutrophil count (ANC) \>= 1,000/mm\^3 * Platelet count \>= 100,000/mm\^3 * Hemoglobin \>= 8 g/dL * Calculated (Calc.) creatinine clearance \>= 30 mL/min using the Cockcroft-Gault equation * Total serum bilirubin =\< 1.5 x upper limit of normal (ULN) * Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 2.5 x ULN (or =\< 5 x ULN for patients with liver metastases or Gilbert's disease) * Urine protein creatinine (UPC) ratio =\< 1 or 24-hour protein \< 1 g * Physicians should consider whether any of the following may render the patient inappropriate for this protocol: * Psychiatric illness which would prevent the patient from giving informed consent. * Uncontrolled medical conditions which, in the opinion of the treating physician, would make this protocol unreasonably hazardous for the patient. * Patients who cannot swallow oral formulations of the agent(s). In addition: * Women and men of reproductive potential should agree to use an appropriate method of birth control throughout their participation in this study due to the teratogenic potential of the therapy utilized in this trial. Include as applicable: Appropriate methods of birth control include abstinence, oral contraceptives, implantable hormonal contraceptives or double barrier method (diaphragm plus condom). * Patients with rheumatoid arthritis and other rheumatologic arthropathies, Sjogren's syndrome and psoriasis controlled with topical medication and or steroids equivalent to \< 10 mg prednisone daily, not on immunosuppressive medications and patients with positive serology are eligible. Patients with vitiligo, endocrine deficiencies including hypo or hyper thyroid disease managed with replacement, diabetes type 1 are eligible. * Sexually active subjects (men and women) must agree to use medically accepted barrier methods of contraception (e.g., male or female condom) during the study and continue for 4 months after the last dose of study drugs, even if oral contraceptives are also used.