Ipilimumab With or Without Sargramostim in Treating Patients With Stage III or Stage IV Melanoma That Cannot Be Removed by Surgery

Inclusion Criteria: * All sites of disease must be evaluated within 4 weeks prior to randomization; patients must have measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) * No more than one prior systemic therapeutic regimen for unresectable stage III or stage IV melanoma; this includes chemotherapy, biologic therapy, biochemotherapy, or investigational treatment; this does not include any therapies given in the adjuvant setting * Histologic diagnosis of metastatic melanoma; for unknown primary disease, diagnosis of metastatic disease by cytology fine needle aspiration (FNA) is not acceptable * Women must not be pregnant or breast-feeding due to unknown effects of ipilimumab and GM-CSF on the unborn fetus; all women of childbearing potential must have a blood test within 72 hours prior to randomization to rule out pregnancy; women of childbearing potential and sexually active males must be strongly advised to use an accepted and effective method of contraception; women of childbearing potential (WOCBP) must be using an adequate method of contraception to avoid pregnancy throughout the study and for up to 12 weeks after the last dose of investigational product, in such a manner that the risk of pregnancy is minimized; sexually mature females who have not undergone a hysterectomy or who have not been postmenopausal naturally for at least 24 consecutive months (i.e., who have had menses at some time in the preceding 24 consecutive months) are considered to be of childbearing potential; women who are using oral contraceptives, other hormonal contraceptives (vaginal products, skin patches, or implanted or injectable products), or mechanical products such as an intrauterine device or barrier methods (diaphragm, condoms, spermicides) to prevent pregnancy, or are practicing abstinence or where their partner is sterile (e.g., vasectomy) should be considered to be of childbearing potential * White blood cells (WBC) \>= 2000/uL (obtained =\< 4 weeks prior to randomization) * Absolute neutrophil count (ANC) \>= 1500/mcL (obtained =\< 4 weeks prior to randomization) * Platelets \>= 100,000/mcL (obtained =\< 4 weeks prior to randomization) * Hemoglobin \>= 8 g/dL (obtained =\< 4 weeks prior to randomization) * Creatinine =\< 3.0 x upper limit of normal (ULN) (obtained =\< 4 weeks prior to randomization) * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) =\< 2.5 x ULN (obtained =\< 4 weeks prior to randomization) * Bilirubin =\< 3.0 x ULN, (except patients with Gilbert's syndrome, who must have a total bilirubin less than 3.0 mg/dL) (obtained =\< 4 weeks prior to randomization) * No concomitant therapy with any of the following: interleukin (IL) 2, interferon, or other non-study immunotherapy regimens; cytotoxic chemotherapy; immunosuppressive agents; other investigation therapies; or chronic use of systemic corticosteroids; must have been discontinued \>= 4 weeks * No infection with human immunodeficiency virus (HIV); due to the mechanism of action of ipilimumab and GM-CSF, activity and side effects in an immune compromised patient are unknown * No active infection with hepatitis B * No active or chronic infection with hepatitis C * Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1 Exclusion Criteria: * Patients with any history of central nervous system (CNS) metastases are excluded * Patients are excluded if they have a history of any other malignancy from which the patient has been disease-free for less than 2 years, with the exception of adequately treated and cured basal or squamous cell skin cancer, superficial bladder cancer or carcinoma in situ of the cervix * Patients are excluded if they have a history of any autoimmune disease; patients with a history of autoimmune thyroiditis are eligible if their current thyroid disorder is treated and stable with replacement or other medical therapy * Patients are excluded for any underlying medical or psychiatric condition, which in the opinion of the investigator, will make the administration of study drug hazardous or obscure the interpretation of adverse events, such as a condition associated with frequent diarrhea * Patients are excluded for receiving any non-oncology vaccine therapy used for prevention of infectious diseases for up to four weeks (28 days) prior to or after any dose of ipilimumab * Patients are excluded if they have a history of prior treatment with ipilimumab or prior cluster of differentiation (CD)137 agonist or cytotoxic T-lymphocyte antigen 4 (CTLA-4) inhibitor or agonist * Any concurrent medical condition requiring the use of systemic steroids is not permitted (the use of inhaled or topical steroids is permitted)