Fulvestrant and/or Anastrozole in Treating Postmenopausal Patients With Stage II-III Breast Cancer Undergoing Surgery

Inclusion Criteria: 1. Female ≥18 years of age 2. Eastern Cooperative Oncology Group (ECOG) performance status 0-2 3. Postmenopausal, verified by: * post bilateral surgical oophorectomy or * no spontaneous menses ≥ 1 year or * no menses for \< 1 year with follicle-stimulating hormone (FSH) and estradiol levels in postmenopausal range, according to institutional standards 4. Pathologic confirmation of invasive breast cancer diagnosed by core needle biopsy 5. Clinical T2-T4c, any N, M0 invasive breast cancer, by AJCC 7th edition clinical staging, with the goal being surgery to complete excision of the tumor in the breast and the lymph node. Primary tumor must be: * palpable * its largest diameters is at least 2.0 cm by physical examination or by radiological assessment Note: * Patients with contralateral ductal carcinoma in situ and/or invasive breast cancer are not eligible. * Patients with multi-centric breast cancer (defined as more than one lesion is invasive breast cancer in the same breast separated by ≥ 2 cm of normal breast tissue are not eligible. 6. Invasive breast cancer is estrogen receptor positive with an Allred score of 6, 7 or 8 by local institution standard protocol. If an Allred Score is not reported on the diagnostic pathology report, ER positivity in \> 66% cells is eligible. If ER positivity is ≤ 66%, the staining intensity (weak, intermediate, strong) is needed to calculate the Allred Score to determine eligibility. 7. Invasive breast cancer is Human Epidermal Growth Factor Receptor 2 (HER2)- A patient is considered to have HER2 negative breast cancer if one of the following applies: * 0 or 1+ by immunohistochemistry (IHC) and ISH not done * 0 or 1+ by IHC and ISH ratio (HER2 gene copy/chromosome 17) \< 2 * 2+ by IHC and ISH ratio (HER2 gene copy/chromosome 17) \< 2 8. Documentation of mammogram and ultrasound (including ductal carcinoma in situ (DCIS) and invasive cancer) of the diseased breast performed within 56 days prior to registration. Mammogram for the unaffected contralateral breast is required within 12 months prior to registration. 9. Laboratory values (≤ 14 days prior to registration): 1. Absolute Neutrophil Count (ANC) \> 1000/mm\^3 2. Platelet Count \> 100,000/mm\^3 3. Total Bilirubin \< 1.5 x upper limits of normal (ULN) 4. Creatinine \< 1.5 x ULN 5. Serum alanine transaminase (ALT) \< 2.5 x ULN 10. Tissue acquisition: Patient must agree to provide the required research biopsies at baseline, week 4 and at surgery for integral and integrated biomarker and correlative studies. Exclusion Criteria: 1. Premenopausal status 2. Inflammatory breast cancer defined as clinically significant erythema of the breast and/or documented dermal lymphatic invasion (not direct skin invasion by tumor or peau d' orange without erythema). 3. An excisional biopsy of this breast cancer. 4. Hormone replacement therapy of any type, megestrol acetate, or raloxifene within one week prior to registration. 5. Tumor estrogen receptor (ER) Allred score between 0-5 or HER2 positive by IHC (3+) or amplified by FISH \> 2.0. 6. Surgical axillary staging procedure prior to study entry. Note: Fine needle aspiration (FNA) or core needle biopsy of axillary node is permitted. 7. Clinical or radiographic evidence of metastatic disease. Metastatic workup is not required, but is recommended for patients with clinical stage III disease. Note: Isolated ipsilateral supraclavicular node involvement is permitted. 8. Breast implants are contraindicated only if the implant precludes the required research biopsies or interferes with palpating the breast lesion. 9. Treatment for this cancer including surgery, radiation therapy, chemotherapy, biotherapy, hormonal therapy or investigational agent prior to study entry. 10. History of invasive breast cancer or contralateral DCIS.