Commit graph

428 commits

Author SHA1 Message Date
thiemowmde
983c157e6b upload: Use more compact ?? operators and such
This makes the code more compact and more readable while doing the
same as before.

One relevant note: I'm also removing a null check for a variable
that will be used as an array index. This is fine because of the ??
operator. What actually happens is an $maxUploadSize[''] array
access, which never exists and falls back to what comes after the ??.

Change-Id: I7fc82fd179c9594ce5755327523ceec4f502d14f
2024-08-08 16:04:05 +02:00
Ebrahim Byagowi
fab78547ad Add namespace to the root classes of ObjectCache
And deprecated aliases for the the no namespaced classes.

ReplicatedBagOStuff that already is deprecated isn't moved.

Bug: T353458
Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
2024-07-10 00:14:54 +03:30
Umherirrender
472891385d Use namespaced classes (2)
Changes to the use statements done automatically via script
Addition of missing use statement done manually

Change-Id: Id9f3e775e143d1a17b6b96812a8230cfba14d9d3
2024-06-16 20:23:55 +02:00
Ebrahim Byagowi
d21cc67450 Add namespace and deprecation alias to FileBackend
This patch introduces a namespace declaration for the
Wikimedia\FileBackend to FileBackend and establishes a class
alias marked as deprecated since version 1.43.

Bug: T353458
Change-Id: Id897687b1d679fd7d179e3a32e617aae10ebff33
2024-05-19 22:35:58 +03:30
Giuseppe Lavagetto
cfa7ed13b1 Switch Special:Upload to use async upload-by-url
With this change, when async uploads are enabled, upload-by-url
will spawn a job and a form with a button to check the status of the
process is shown to the user.

In the process, add processing of warnings in the remote jobs spawned by
the API or the Special page. This is done by adding checks to
UploadJobTrait::verifyUpload. In order to manage warnings serialized in
the job status, a method to unserialize the result of
UploadBase::makeWarningsSerializable.

Things that we might want to fix:
* The form's UI is abysmal, we should probably use Codex
* While it's not a huge deal, I'd like to figure out why I need to
purge the page cache if I want the file to show up. And more
interestingly, why this doesn't happen when uploading via the API

Bug: T295007
Bug: T118887
Change-Id: I49181d93901f064815808380285fc4abae755341
2024-03-28 11:01:46 +01:00
Giuseppe Lavagetto
b5ed16c1e7 Allow async upload by url via the Api
To this end if 'async' is passed with 'url' to the api:
* Avoid downloading the file synchronously, but verify early
  if the upload is allowed by adding a canFetchFile to UploadBase
  overridden in UploadFromUrl
* Spawn an UploadFromUrlJob
* When checking for the status of the job, do it fetching the data in
  the main stash.

Bug: T295007
Change-Id: If95ccf376cfa9fbe9b3cd058e8e334a6bdd2eb44
2024-03-23 11:23:07 +01:00
Brian Wolff
15c3fb401a Make verifyPartialFile reuse SHA1 hash if we already know
During a stashed upload, the SHA1 has already been calculated and
is populated based on data saved in DB. Reuse that value in
verifyPartialFile() instead of recalculating as SHA1 can take a
long time to calculate for large files.

This should improve the speed of PublishStashedFile jobs.

Bug: T200820
Change-Id: Ie2967c636b2f942921a125ef62d1a466c6035ca0
2024-03-02 00:56:39 -08:00
Brian Wolff
befd8fcd93 Optimize AssembleUploadChunks to reuse SHA1 hash instead of recalc
AssembleUploadChunks was calculating the SHA1 hash of the same
file 5 times in a row. Calculating SHA1 hashes can be somewhat
expensive for multi-GB files, making the job slow, possibly to
the point of a timeout. This change ensures that the SHA1 value
is kept and reused when applicable so that the job will only
calculate it once.

Bug: T200820
Change-Id: I842814c7a2b7dc6e427e040c8dd4d43e7c7cabb4
2024-02-25 23:54:33 -08:00
Brian Wolff
bb0209d56e Add additional debug logging for chunked upload
Users often complain that chunked upload is unreliable. However
it is often difficult to see what happened when it failed. Add
additional debug logging so we can better determine how often
chunked upload fails, and hopefully have a better idea what the
causes are.

This only adds logging and should not change any behaviour

Change-Id: I45b710fa57c7d05bb27a7b00a3303e78f5d2ff2a
2024-02-16 15:55:39 -08:00
Brian Wolff
ea84b992ef Change $wgSVGMetadataCutoff default to 5 MiB (previously 512KiB).
This is used to (among other things) detect lang tags in multilingual
SVGs. Users have complained that lang tags are often missed in large
SVG files.

The cut-off is used for two things during upload:
* Run some (simple) regexes to detect <?xml header
* Use XMLReader (with entity substitution enabled!) to look for specific tags.

The first check doesn't make sense to use a configurable cut off. Change
it to look at the first 4096 bytes only. The <?xml header is required to be
the first thing in the file other than BOM, so this should be more than
sufficient. XML parsers give a fatal error if there is whitespace before
the <?xml declaration.

It seems unlikely to be problematic to use XMLReader on up to 5MB of the file,
since that is a "pull" XML parser, and won't load the entire file at once.
The code that cuts off the SVG at the 5MB mark likely uses more memory
than parsing the file does. In fact, we separately use XMLReader to do
security checks with no such cut-off, so potentially it could even make sense
to remove the cut-off entirely, since clearly parsing the full file is not
causing problems.

Bug: T270889
Change-Id: I7350918647d92c40934a7c86e906b6bfb8a40ada
2024-02-09 19:20:34 -08:00
Brooke Vibber
dcd9c3ae26 Update name & email for bvibber
Updating name & email addresses for Brooke Vibber.

Re-ran updateCredits.php as well so there are some new entries in
there as well.

There are a couple of files in resources/libs that will have to
be changed upstream to keep tests happy, I will do patches
later. :D

Change-Id: I2f2e75d3fa42e8cf6de19a8fbb615bac28efcd54
2024-02-08 17:02:16 -08:00
James D. Forrester
4bae64d1c7 Namespace includes/context
Bug: T353458
Change-Id: I4dbef138fd0110c14c70214282519189d70c94fb
2024-02-08 11:07:01 -05:00
jenkins-bot
51ef54982f Merge "upload: Remove a duplicate strpos call in UploadBase" 2024-01-25 17:20:13 +00:00
thiemowmde
69d17d7659 upload: Remove a duplicate strpos call in UploadBase
Just use the result from one call in both places.

I'm also re-arranging the code for readability. This is quite
critical here. A file name like ".htaccess" where the very first
character is a dot but no other dot follows must be considered a
filename without an extension. I hope this is more visible with
the `> 0`.

Change-Id: I24179de62c3f4443effe8a4ebd089a3f77fd84e3
2024-01-25 12:26:06 +01:00
Amir Sarabadani
014bc61006 Remove more indirect calls to IDBAccessObject::READ_* constants
Found via (?<!IDBAccessObject)::READ_

We are planning to deprecate and remove implementing IDBAccessObject
interface just to use the constants.

Bug: T354194
Change-Id: I89d442fa493b8e5332ce118e5bf13f13b8dd3477
2024-01-23 15:42:38 +01:00
Daimona Eaytoy
175c0c4abf Replace more instances of deprecated MWException
Bug: T328220
Change-Id: Iba90f7f9b5766bccc05380d040138d74d5e9558a
2024-01-19 23:11:59 +00:00
daniel
2cb8d6fbde layering: UploadBase should not depend on API modules.
API modules are high level request handler, lower level code should not
depend on them.

This patch solves the problem only partially, since it leaves references
to ApiUpload in AssembleUploadChunksJob and PublishStashedFileJob. These
jobs were already accessing ApiMain, so while this does not fully resolve
the problem, it reduces it.

Change-Id: I39c9e30cfb2860c573eed8a791f1a292a83cbd76
2023-12-16 01:29:45 +00:00
daniel
d5cc98f9ff Deprecate UploadBase::isThrottled
This method is now redundant since rate limit checks are implicit in
permission checks. verifyPermissions() calls authorizeWrite( 'upload' ),
which will enforce any limits on the upload action.

Change-Id: I2ab3c646b8246411df501b548f652eaf11d0bc8e
2023-10-23 08:43:14 +00:00
James D. Forrester
468e69bccc Namespace Sanitizer under \MediaWiki\Parser
Bug: T166010
Change-Id: Id13dcbf7a0372017495958dbc4f601f40c122508
2023-09-21 05:39:23 +00:00
James D. Forrester
1d0b7ae1e2 Namespace User under \MediaWiki\User
Bug: T166010
Change-Id: I7257302b485588af31384d4f7fc8e30551f161f1
2023-09-19 19:18:16 +00:00
jenkins-bot
d6f99f92cd Merge "upload: Fix doc on UploadVerifyUploadHook" 2023-09-19 13:04:52 +00:00
Umherirrender
51905715d6 upload: Fix doc on UploadVerifyUploadHook
Doc comment indicates the variable is only used for new uploads,
explicit mention the false type used on reupload.

Bug: T315521
Change-Id: I47bbfdeb327b26b1575b9c649cb6b0325390c9a2
2023-09-12 19:34:42 +02:00
Amir Sarabadani
5bd33d46ef Reorg: Move WebRequest to includes\Request
This has been approved as part of RFC T166010

Bug: T321882
Change-Id: I6bbdbbe6ea48cc1f50bc568bb8780fc7c5361a6f
2023-09-11 21:44:34 +01:00
Daimona Eaytoy
6ddbd53eda Remove redundant empty() constructs (3)
empty() only makes sense when the expression it checks is possibly
undefined, otherwise it's equivalent to a truthiness check with the
additional downside of suppressing errors when it's not wanted.

Replace it with simple truthiness checks, using strict comparison when
that seems to help with polymorphic variables.

These were caught by a bespoke phan plugin.

Change-Id: Ide262162553d2da7e5388d05e8731529c44591c1
2023-09-08 23:37:23 +02:00
Amir Sarabadani
f4e68e055f Reorg: Move Status to MediaWiki\Status\
This class is used heavily basically everywhere, moving it to Utils
wouldn't make much sense. Also with this change, we can move
StatusValue to MediaWiki\Status as well.

Bug: T321882
Depends-On: I5f89ecf27ce1471a74f31c6018806461781213c3
Change-Id: I04c1dcf5129df437589149f0f3e284974d7c98fa
2023-08-25 15:44:17 +02:00
Umherirrender
be67b9ba05 build: Remove unneeded newline from one line condition/foreach
Improve readability

Change-Id: I22ed08bbd8d821c4d4df620af4c5e97b043c305a
2023-08-22 21:49:58 +02:00
Reedy
0b7f18be63 UploadBase: Minor cleanup
Change-Id: I5435aa1f79c1504ca306e7215b9dcbbbb7d9df45
2023-08-08 00:41:30 +00:00
Reedy
e332b99669 UploadBase: Improve wording in comments
Change-Id: I2469818bedb6cd97345e220e35cec2849a056abf
2023-07-24 20:16:04 +00:00
thiemowmde
9b03cde58e Merge sequences of if that end doing the same thing anyway
Motivation:
* Avoid code duplication.
* Hopefully make it easier to read.
* Also order stuff from cheap to expensive, if possible.

Change-Id: I575e3f2027ce60a0d0885be5b9bd3e07bc035eee
2023-06-16 16:09:42 +02:00
jenkins-bot
6f9ac8889b Merge "Replace substr with cleaner string methods" 2023-06-06 14:15:27 +00:00
Umherirrender
0ed462e19a hook: Document null type for pass-by-ref on UploadVerifyUploadHook
Similiar to $error on UploadStashFileHook (0b7295a)

Change-Id: I2637915f04477a37e865b772008dcba1aa0da24a
2023-06-04 21:10:25 +02:00
Matěj Suchánek
676fcf4379 Replace substr with cleaner string methods
Use str_starts_with, str_ends_with or string offset where appropriate.

This fixes a bug in MimeAnalyzer where the "UTF-16LE" header could not
be identified because of wrong constant. This is the exact type of bug
that the new functions can avoid.

Change-Id: I9f30881e7e895f011db29cf5dcbe43bc4f341062
2023-05-20 15:40:21 +02:00
Umherirrender
e04d3a28f6 Replace internal Hooks::runner
The Hooks class contains deprecated functions and the whole class is
going to get removed, so remove the convenience function and inline the
code.

Bug: T335536
Change-Id: I8ef3468a64a0199996f26ef293543fcacdf2797f
2023-05-11 06:17:38 +00:00
Tim Starling
be3018b268 Just another 80 or so PHPStorm inspection fixes (#4)
* Unnecessary regex modifier. I agree with this inspection which flags
  /s modifiers on regexes that don't use a dot.
* Property declared dynamically.
* Unused local variable. But it's acceptable for an unused local
  variable to take the return value of a method under test, when it is
  being tested for its side-effects. And it's acceptable for an unused
  local variable to document unused list expansion elements, or the
  nature of array keys in a foreach.

Change-Id: I067b5b45dd1138c00e7269b66d3d1385f202fe7f
2023-03-25 00:39:06 +00:00
James D. Forrester
161dde1536 UploadBase::detectScript: Protect against null inputs to str_starts_with
These trigger PHP 8.1 warnings, though oddly this has been seen on the
REL1_40 branch but not elsewhere yet. Worth doing anyway, each of these
calls is safe to fail on '' instead.

Change-Id: I1fe18f487a5e56effc61bf053055882958603418
2023-03-14 15:29:54 -04:00
Derk-Jan Hartman
005d20e470 Remove IEContentAnalyzer
This supported mostly IE 6 and 7 as well as some very old Safari
versions against sniffing. Browsers later implemented
X-Content-Type-Options: nosniff which should protect against this
problem.

Bug: T309787
Change-Id: Iea61bd82033551008eb1832fe899ea28ccb23385
2023-03-07 02:27:45 +00:00
jenkins-bot
7e12aab9d3 Merge "UploadBase::checkFileExtension: Guard for null input triggering PHP 8.1 warning" 2023-03-02 20:16:11 +00:00
James D. Forrester
b33801a03a UploadBase::checkFileExtension: Guard for null input triggering PHP 8.1 warning
Change-Id: I5982243670e86f30b30a212be13a65be408aa19b
2023-03-02 09:38:19 -05:00
James D. Forrester
ad06527fb4 Reorg: Namespace the Title class
This is moderately messy.

Process was principally:

* xargs rg --files-with-matches '^use Title;' | grep 'php$' | \
  xargs -P 1 -n 1 sed -i -z 's/use Title;/use MediaWiki\\Title\\Title;/1'
* rg --files-without-match 'MediaWiki\\Title\\Title;' . | grep 'php$' | \
  xargs rg --files-with-matches 'Title\b' | \
  xargs -P 1 -n 1 sed -i -z 's/\nuse /\nuse MediaWiki\\Title\\Title;\nuse /1'
* composer fix

Then manual fix-ups for a few files that don't have any use statements.

Bug: T166010
Follows-Up: Ia5d8cb759dc3bc9e9bbe217d0fb109e2f8c4101a
Change-Id: If8fc9d0d95fc1a114021e282a706fc3e7da3524b
2023-03-02 08:46:53 -05:00
jenkins-bot
1ec7ef8fe4 Merge "upload: Allow attributes starting with "on" in inkscape SVG namespace" 2023-02-13 00:58:17 +00:00
Derk-Jan Hartman
c64a8faab6 upload: Allow attributes starting with "on" in inkscape SVG namespace
Inkscape has an attribute only-selected in it's namespace which
is not interpreted by browsers and should be safe upload.

Bug: T288186
Change-Id: I044f225aef813efc708eb96f588fc55733d165f3
2023-02-13 00:41:22 +00:00
jenkins-bot
6a25db1206 Merge "Change $wgLang to use Message::*Param functions" 2023-01-28 00:31:50 +00:00
jenkins-bot
925834518d Merge "Code style cleanup" 2023-01-06 17:20:09 +00:00
jenkins-bot
2cff056b94 Merge "UploadBase.php: added namespace for w3c-test-suite" 2023-01-05 21:49:03 +00:00
Derk-Jan Hartman
a5e0700da3 Code style cleanup
Some minor changes to silence the IDE

- Removed some unnecessary branching
- Use strict comparison where possible
- Use self::

Change-Id: Ica98f2d02520c9537f71dfd517a397828bf68848
2023-01-05 21:15:30 +00:00
Umherirrender
45e6a2b0f9 Use str_starts_with/str_ends_with/str_contains
Use the new function in conditions to avoid creating substrings or to
search the whole string

Change-Id: Ibad6b1b447a4f62cceb34359231f88ebb967a90b
2022-12-12 19:54:24 +01:00
Umherirrender
1b342a8893 Various doc fixes about false and null on method arguments/return types
Doc-only changes

Change-Id: Ice974b3ba41708859dfe646e94b31c5ebbf26410
2022-11-03 18:55:47 +01:00
Zabe
f6b9381d7f Revert "Reorg: Move some of request related classes to MediaWiki/Request"
This reverts commit 2bdc0b2b72.

Reason for revert: T166010#8349431

Bug: T166010
Change-Id: Idcd3025647aec99532f5d69b9c1718c531761283
2022-10-27 13:14:16 +00:00
Amir Sarabadani
2bdc0b2b72 Reorg: Move some of request related classes to MediaWiki/Request
Moving:
 - DerivativeRequest
 - FauxRequest
 - FauxRequestUpload
 - PathRouter
 - WebRequest
 - WebRequestUpload

Bug: T166010
Change-Id: I5ea70120d745f2876ae31d039f3f8a51e49e9ad8
2022-10-26 16:49:10 +02:00
Tim Starling
0077c5da15 Use short array destructuring instead of list()
Introduced in PHP 7.1. Because it's shorter and looks nice.

I used regex replacement.

Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
2022-10-21 15:33:37 +11:00