laramies/theHarvester
1
1
Fork 0
mirror of https://github.com/laramies/theHarvester.git synced 2025-02-25 06:53:05 +08:00
Code Issues Projects Releases Packages Wiki Activity
3768 commits 12 branches 26 tags 10 MiB
Commit graph

3768 commits

This branch
This branch
All branches
Author SHA1 Message Date
dependabot[bot]
1158fd10a3 Bump aiohttp from 3.11.2 to 3.11.6 
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.11.2 to 3.11.6.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.11.2...v3.11.6)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-20 02:52:55 +00:00
dependabot[bot]
ade8764bfe Bump wheel from 0.44.0 to 0.45.0 
Bumps [wheel](https://github.com/pypa/wheel) from 0.44.0 to 0.45.0.
- [Release notes](https://github.com/pypa/wheel/releases)
- [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst)
- [Commits](https://github.com/pypa/wheel/compare/0.44.0...0.45.0)

---
updated-dependencies:
- dependency-name: wheel
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-17 08:07:14 +00:00
dependabot[bot]
5e4b935914 Bump aiohttp from 3.10.10 to 3.11.2 
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.10.10 to 3.11.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.10.10...v3.11.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-17 08:06:54 +00:00
dependabot[bot]
2949243ad4 Bump fastapi from 0.115.4 to 0.115.5 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.115.4 to 0.115.5.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.115.4...0.115.5)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-17 08:06:36 +00:00
dependabot[bot]
8111bdc4cb Bump ruff from 0.7.3 to 0.7.4 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.7.3 to 0.7.4.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.7.3...0.7.4)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-17 08:04:42 +00:00
dependabot[bot]
16cc8afc8d Bump ruff from 0.7.2 to 0.7.3 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.7.2...0.7.3)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-10 21:17:46 +00:00
dependabot[bot]
25ca73c8ef Bump ruff from 0.7.1 to 0.7.2 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.7.1...0.7.2)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-11-01 22:07:36 +00:00
dependabot[bot]
17ecd4f40b Bump fastapi from 0.115.3 to 0.115.4 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.115.3 to 0.115.4.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.115.3...0.115.4)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-28 22:33:56 +00:00
J.Townsend
7457d6fb85
Fix mypy 2024-10-27 01:46:06 +00:00
J.Townsend
dadf4a3180
Fix mypy check 2024-10-27 01:35:35 +00:00
J.Townsend
ddd45d7cf4
Remove broken option 2024-10-27 01:17:35 +00:00
L1ghtn1ng
4cf270724e Upgrade to Python 3.13 and enhance Dockerfile setup 
Update base image to Python 3.13 and adjust pipx installation accordingly. Add curl and gcc installations, and ensure bashrc sourcing for environment setup.
 2024-10-27 01:12:37 +00:00
L1ghtn1ng
5b83e45a8a Update pipx install to specify Python version 
Specifying Python 3.12 for pipx installation ensures compatibility and proper functionality of the installed package. This change avoids any default Python version conflicts that might arise during the build process.
 2024-10-27 01:47:55 +01:00
L1ghtn1ng
edfdf5fa74 Update Docker Python version , adjust Docker ignore files 
Upgraded Docker image to Python 3.12 and changed code quality tool references in CONTRIBUTING.md from lgtm to ruff. Also adjusted .dockerignore to include new files and remove outdated config files.
 2024-10-27 01:09:59 +01:00
L1ghtn1ng
13a0cdd62a Add async pagination helper and update pytest settings 
Introduce the `next_page_or_end` method in `githubcode.py` to handle pagination. Additionally, modify the test case and upgrade pytest settings in `pyproject.toml` to the latest versions, ensuring compatibility and updated configurations.
 2024-10-27 01:02:05 +01:00
L1ghtn1ng
c18012d50a Refactor Intelx and GitHubCode classes for async HTTP requests 
Refactor the `intelxsearch.py` to use `aiohttp` for asynchronous HTTP requests instead of `requests` and `ujson`. This improves speed and resource management in the `do_search` method. Additionally, improve the `get_interestingurls` method to parse and return subdomains. In `githubcode.py`, reformat the `fragments_from_response` method for better readability.
 2024-10-26 23:10:26 +01:00
L1ghtn1ng
e2acaa099e Add error handling to GitHub code search methods 
Refactor GitHub code search to include comprehensive try-except blocks across methods. This enhances robustness by capturing and logging exceptions, ensuring the system fails gracefully and provides meaningful error messages.
 2024-10-26 20:18:04 +01:00
L1ghtn1ng
0b0d6516b1 Update RocketReach API query parameters and pagination logic 
Revised the data query parameters to use 'current_employer_domain' instead of 'company_domain' and updated the pagination handling to align with the new API response structure. Enhanced exception logging to specify RocketReach as the source of the error.
 2024-10-26 19:27:49 +01:00
J.Townsend
4495dc8407
Update README.md 2024-10-26 17:56:31 +01:00
J.Townsend
88135de5bf
fix typo 2024-10-26 17:54:56 +01:00
dependabot[bot]
e6d4755920 Bump ruff from 0.7.0 to 0.7.1 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.7.0...0.7.1)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-24 23:02:17 +01:00
dependabot[bot]
632808dab4 Bump mypy from 1.12.1 to 1.13.0 
Bumps [mypy](https://github.com/python/mypy) from 1.12.1 to 1.13.0.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/python/mypy/compare/v1.12.1...v1.13.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-23 22:15:43 +01:00
dependabot[bot]
985c49d7ba Bump fastapi from 0.115.2 to 0.115.3 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.115.2 to 0.115.3.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.115.2...0.115.3)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-22 22:13:39 +01:00
dependabot[bot]
d2c5f8a23c Bump censys from 2.2.15 to 2.2.16 
Bumps [censys](https://github.com/censys/censys-python) from 2.2.15 to 2.2.16.
- [Release notes](https://github.com/censys/censys-python/releases)
- [Commits](https://github.com/censys/censys-python/compare/v2.2.15...v2.2.16)

---
updated-dependencies:
- dependency-name: censys
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-21 23:38:46 +01:00
dependabot[bot]
4ac782e0ec Bump mypy from 1.12.0 to 1.12.1 
Bumps [mypy](https://github.com/python/mypy) from 1.12.0 to 1.12.1.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/python/mypy/compare/v1.12.0...v1.12.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-21 23:38:22 +01:00
dependabot[bot]
7475c4f6e7 Bump playwright from 1.47.0 to 1.48.0 
Bumps [playwright](https://github.com/Microsoft/playwright-python) from 1.47.0 to 1.48.0.
- [Release notes](https://github.com/Microsoft/playwright-python/releases)
- [Commits](https://github.com/Microsoft/playwright-python/compare/v1.47.0...v1.48.0)

---
updated-dependencies:
- dependency-name: playwright
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-21 23:37:53 +01:00
dependabot[bot]
b82f2f9990 Bump types-requests from 2.32.0.20240914 to 2.32.0.20241016 
Bumps [types-requests](https://github.com/python/typeshed) from 2.32.0.20240914 to 2.32.0.20241016.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-18 10:28:43 +01:00
dependabot[bot]
faa7ec3839 Bump censys from 2.2.14 to 2.2.15 
Bumps [censys](https://github.com/censys/censys-python) from 2.2.14 to 2.2.15.
- [Release notes](https://github.com/censys/censys-python/releases)
- [Commits](https://github.com/censys/censys-python/compare/v2.2.14...v2.2.15)

---
updated-dependencies:
- dependency-name: censys
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-18 10:28:29 +01:00
dependabot[bot]
4a16d1f307 Bump ruff from 0.6.9 to 0.7.0 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.6.9 to 0.7.0.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.6.9...0.7.0)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-18 10:28:14 +01:00
L1ghtn1ng
8795163e16 Update version to 4.7.0 
Incremented the software version from 4.6.0 to 4.7.0 in version.py.
 2024-10-16 06:07:35 +01:00
L1ghtn1ng
cc1c653ad4 Move continue-on-error for Python 3.13 to individual steps 
Shifted the `continue-on-error` condition for Python 3.13 from the job level to each individual step within the GitHub Actions workflow. This ensures more granular control over error handling, allowing other steps to succeed or fail independently.
 2024-10-16 05:51:41 +01:00
L1ghtn1ng
40ecb55c41 Revise error handling in CI workflow 
Consolidate error handling by setting `continue-on-error` at the job level with a condition for Python 3.13. Remove redundant `continue-on-error` directives from individual steps to streamline the workflow file.
 2024-10-16 05:29:30 +01:00
L1ghtn1ng
014ed70e1e ci tweak 2024-10-16 05:08:37 +01:00
L1ghtn1ng
d4162486a4 ci tweak 2024-10-16 05:03:56 +01:00
L1ghtn1ng
dd3bef6c40 Increase parallelism and add continue-on-error to steps 
Updated the max-parallel setting from 8 to 10 to increase workflow efficiency. Also added the continue-on-error parameter to multiple steps to ensure the workflow continues despite errors. This improves the robustness and speed of the CI process.
 2024-10-16 04:59:00 +01:00
L1ghtn1ng
5b2c453c6f fix linting 2024-10-16 04:47:19 +01:00
L1ghtn1ng
62277a37ba Add Python 3.13 to GitHub workflow 
Extend the CI matrix to include Python 3.13 alongside existing versions. This ensures compatibility and automated testing for the latest Python release.
 2024-10-16 04:31:57 +01:00
L1ghtn1ng
2c7b2bb249 Fix proxy handling and re-enable mypy type checking as tests fixed 
Updated aiohttp calls to handle proxies correctly by converting them to strings when necessary. Removed an unnecessary blank line in core.py, and re-enabled static type checking with mypy in the GitHub workflow now that mypy tests are passing
 2024-10-16 04:27:20 +01:00
dependabot[bot]
4f601d72f4 Bump uvloop from 0.20.0 to 0.21.0 
Bumps [uvloop](https://github.com/MagicStack/uvloop) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/MagicStack/uvloop/releases)
- [Commits](https://github.com/MagicStack/uvloop/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: uvloop
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-16 03:41:08 +01:00
dependabot[bot]
e4c8d6c7f1 Bump types-python-dateutil from 2.9.0.20240906 to 2.9.0.20241003 
Bumps [types-python-dateutil](https://github.com/python/typeshed) from 2.9.0.20240906 to 2.9.0.20241003.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-python-dateutil
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-16 03:40:33 +01:00
dependabot[bot]
fcc0a47fb8 Bump aiohttp from 3.10.8 to 3.10.10 
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.10.8 to 3.10.10.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.10.8...v3.10.10)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-16 03:40:14 +01:00
dependabot[bot]
f1a9025fed Bump ruff from 0.6.8 to 0.6.9 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.6.8 to 0.6.9.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.6.8...0.6.9)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-16 03:39:34 +01:00
dependabot[bot]
6dff49e916 Bump mypy from 1.11.2 to 1.12.0 
Bumps [mypy](https://github.com/python/mypy) from 1.11.2 to 1.12.0.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/python/mypy/compare/v1.11.2...v1.12.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-16 03:39:03 +01:00
L1ghtn1ng
7550063c17 Update Python version requirements and clean up code 
Upgraded the minimum required Python version to 3.11 across various files. Removed `pyre-check` from dependencies and temporarily disabled mypy static type checker in the CI configuration. Additionally, simplified several condition checks and fixed formatting inconsistencies in theHarvester modules.
 2024-10-16 03:27:09 +01:00
dependabot[bot]
319c1e5d1d Bump uvicorn from 0.31.0 to 0.32.0 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.31.0 to 0.32.0.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.31.0...0.32.0)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-16 01:25:50 +01:00
dependabot[bot]
46df4d000f Bump fastapi from 0.115.0 to 0.115.2 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.115.0 to 0.115.2.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.115.0...0.115.2)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-15 08:27:42 +01:00
dependabot[bot]
e25c269312 Bump dnspython from 2.6.1 to 2.7.0 
Bumps [dnspython](https://github.com/rthalley/dnspython) from 2.6.1 to 2.7.0.
- [Release notes](https://github.com/rthalley/dnspython/releases)
- [Changelog](https://github.com/rthalley/dnspython/blob/main/doc/whatsnew.rst)
- [Commits](https://github.com/rthalley/dnspython/compare/v2.6.1...v2.7.0)

---
updated-dependencies:
- dependency-name: dnspython
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-10-08 18:43:31 +01:00
dependabot[bot]
5bf0d75e5b Bump aiohttp from 3.10.6 to 3.10.8 
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.10.6 to 3.10.8.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.10.6...v3.10.8)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-09-30 23:01:31 +01:00
dependabot[bot]
2cd6435611 Bump uvicorn from 0.30.6 to 0.31.0 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.30.6 to 0.31.0.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.30.6...0.31.0)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-09-28 04:58:07 +01:00
dependabot[bot]
e9d8b52e9e Bump types-requests from 2.32.0.20240907 to 2.32.0.20240914 
Bumps [types-requests](https://github.com/python/typeshed) from 2.32.0.20240907 to 2.32.0.20240914.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-09-27 13:50:21 +01:00