In a large pot, heat oil over medium-high. Add celery, carrots, and onion and cook until celery and carrots are crisp-tender, about 5 minutes; season with salt and pepper. Add broth and 4 cups water and bring to a boil. Add chicken, reduce to a simmer, and cover. Cook until chicken is cooked through, about 10 minutes. With tongs, remove chicken and chop or shred into bite-size pieces. Add pasta to pot and cook until tender, about 4 minutes; season with salt and pepper. To serve, stir in chicken and dill.