Drambluker
/
itdoxy-lab
mirror of https://github.com/Drambluker/gitea.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
itdoxy-lab/docs
History
Jason Song 375fd15fbf
Refactor indexer (#25174) 
Refactor `modules/indexer` to make it more maintainable. And it can be
easier to support more features. I'm trying to solve some of issue
searching, this is a precursor to making functional changes.

Current supported engines and the index versions:

| engines | issues | code |
| - | - | - |
| db | Just a wrapper for database queries, doesn't need version | - |
| bleve | The version of index is **2** | The version of index is **6**
|
| elasticsearch | The old index has no version, will be treated as
version **0** in this PR | The version of index is **1** |
| meilisearch | The old index has no version, will be treated as version
**0** in this PR | - |


## Changes

### Split

Splited it into mutiple packages

```text
indexer
β”œβ”€β”€ internal
β”‚Β Β  β”œβ”€β”€ bleve
β”‚Β Β  β”œβ”€β”€ db
β”‚Β Β  β”œβ”€β”€ elasticsearch
β”‚Β Β  └── meilisearch
β”œβ”€β”€ code
β”‚Β Β  β”œβ”€β”€ bleve
β”‚Β Β  β”œβ”€β”€ elasticsearch
β”‚Β Β  └── internal
└── issues
    β”œβ”€β”€ bleve
    β”œβ”€β”€ db
    β”œβ”€β”€ elasticsearch
    β”œβ”€β”€ internal
    └── meilisearch
```

- `indexer/interanal`: Internal shared package for indexer.
- `indexer/interanal/[engine]`: Internal shared package for each engine
(bleve/db/elasticsearch/meilisearch).
- `indexer/code`: Implementations for code indexer.
- `indexer/code/internal`: Internal shared package for code indexer.
- `indexer/code/[engine]`: Implementation via each engine for code
indexer.
- `indexer/issues`: Implementations for issues indexer.

### Deduplication

- Combine `Init/Ping/Close` for code indexer and issues indexer.
- ~Combine `issues.indexerHolder` and `code.wrappedIndexer` to
`internal.IndexHolder`.~ Remove it, use dummy indexer instead when the
indexer is not ready.
- Duplicate two copies of creating ES clients.
- Duplicate two copies of `indexerID()`.


### Enhancement

- [x] Support index version for elasticsearch issues indexer, the old
index without version will be treated as version 0.
- [x] Fix spell of `elastic_search/ElasticSearch`, it should be
`Elasticsearch`.
- [x] Improve versioning of ES index. We don't need `Aliases`:
- Gitea does't need aliases for "Zero Downtime" because it never delete
old indexes.
- The old code of issues indexer uses the orignal name to create issue
index, so it's tricky to convert it to an alias.
- [x] Support index version for meilisearch issues indexer, the old
index without version will be treated as version 0.
- [x] Do "ping" only when `Ping` has been called, don't ping
periodically and cache the status.
- [x] Support the context parameter whenever possible.
- [x] Fix outdated example config.
- [x] Give up the requeue logic of issues indexer: When indexing fails,
call Ping to check if it was caused by the engine being unavailable, and
only requeue the task if the engine is unavailable.
- It is fragile and tricky, could cause data losing (It did happen when
I was doing some tests for this PR). And it works for ES only.
- Just always requeue the failed task, if it caused by bad data, it's a
bug of Gitea which should be fixed.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
 		2023-06-23 12:37:56 +00:00
..
assets/js Update JS dependencies, add new eslint rules (#24597) 2023-05-09 02:35:49 +00:00
content Refactor indexer (#25174) 2023-06-23 12:37:56 +00:00
layouts Remove meta charset from HTML5 documents (#24744) 2023-05-16 05:13:23 +00:00
scripts Run hugo via `go run` and lock its version (#22206) 2022-12-21 21:09:53 -05:00
static Docs for Gitea Actions (#24405) 2023-05-05 20:07:20 +00:00
.gitignore Support Issue forms and PR forms (#20987) 2022-09-02 15:58:49 +08:00
LICENSE Import docs into main repository (#2874) 2017-11-26 23:44:32 +02:00
Makefile Restructure documentation. Now the documentation has installation, administration, usage, development, contributing the 5 main parts (#23629) 2023-03-23 23:18:24 +08:00
README.md Update to docs README (#6221) 2019-03-01 15:39:30 -05:00
README_ZH.md update discord link (#10455) 2020-02-25 12:36:45 -05:00
config.yaml Use Go 1.20 for next release (#24859) 2023-05-22 16:29:53 +00:00

README.md

Gitea: Docs

Join the chat at https://img.shields.io/discord/322538954119184384.svg

Hosting

These pages are hosted using netlifycms and get automatically updated on every push to the master branch.

Install

These pages use the Hugo static site generator. If you are planning to contribute you'll want to download and install Hugo on your local machine.

The installation of Hugo is out of the scope of this document, so please take the official install instructions to get Hugo up and running.

Development

To generate the website and serve it on localhost:1313 just execute this command and stop it with Ctrl+C:

make server

When you are done with your changes just create a pull request, after merging the pull request the website will be updated automatically.

Contributing

Fork -> Patch -> Push -> Pull Request

Authors

License

This project is under the Apache-2.0 License. See the LICENSE file for the full license text.

Copyright

Copyright (c) 2016 The Gitea Authors <https://gitea.io>