Add array_api_dispatch to spmd validation scope (#2940) (#3098)

* Add array_api_dispatch to spmd validation scope

* formatting

* Combine onedal _device_offload.py change

* re-add transform output logic

* black

* minor restoration

* restore host transfer if no queue

* add new array api algos and import config_context

* debugging

* kmeans hotfix

* remove debug

(cherry picked from commit 29ec7ad)

Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

Revert "MAINT: Remove deprecated attributes for BasicStatistics (#2897)"

This reverts commit c760965.

Fix for numpy 2.4 scalar conversion (#2868) (#2875)

(cherry picked from commit 3cc3f4d)

Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

Fix azp docbuild by skipping medium linkcheck (#2830) (#2849)

* Fix azp docbuild by skipping medium linkcheck

* fix incorrect regex

(cherry picked from commit 42096dd)

Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

Add NearestNeighbors SPMD API (#2557)

* Add NearestNeighbors SPMD API

* black format

* extend gold data to have multiple rows per rank

* formatting

* raw inputs support for kneighbors

* Reduce rows of synthetic large test

* update search size and only use _spmd_assert_allclose

* support empty kneighbors()

* Update sklearnex/spmd/neighbors/tests/test_neighbors_spmd.py

* address comments

[CI] upgrade dpctl/dpnp versions (#2414)

* Update build-and-test-lnx.yml

* Update ci.yml

* Update build-and-test-lnx.yml

* Update renovate.json

* Update renovate.json

* Update test_common.py

* Update test_common.py

* Update test_common.py

* Update build-and-test-lnx.yml

* Update build-and-test-lnx.yml

* Update ci.yml

* Update renovate.json

* Update build-and-test-lnx.yml

* Update test_common.py

* Update test_common.py

* Update common.py

* Update coordinate_descent.py

* Update logistic_regression.py

* Update logistic_regression.py

* Update _forest.py

* Update _forest.py

* Update ci.yml

* Update ci.yml

* Update _data_conversion.py

* Update _data_conversion.py

* Update _device_offload.py

* try again

* fixes

* fix KMeans transform issue

* formatting

* further fixes

* Update install_dpcpp.sh

* stopping point

* Update ci.yml

* face facts

* clarification

* add renovatebot checks

* switch source for dpctl/dpnp tracking

* fix potential issue with nightly.yml

* fix bug

* forgotten save

* try and fix CI

* fix method vs staticmethod issue

* try again

* try again

* move validate_data

* Update test_data.py

* reformat

* try again

* try to lazy import changes for dpnp

* try with the spelling correction

* try again

* Update ci.yml

* Update _data_conversion.py

* Update coordinate_descent.py

* Update _data_conversion.py

* Update coordinate_descent.py

* Update _data_conversion.py

* Update ci.yml

* Update ci.yml

* fix issues related to dpctl/dpnp

* further fixes

* final hopefully

* sometime I can be really dumb

* sometime I can be really dumb

* Update _device_offload.py

* fixes and requests

* try to fix spmd use_raw_input issues

* fixes and requests

install scikit-learn to build docs (#2592) (#2593)

Fixes for sklearn 1.7 pre-release support (#2451) (#2527)

* Fixes for sklearn 1.7 pre-release support

* Fix ensemble probabilities interval

* Change scaling method to `clip`

---------


(cherry picked from commit 95c73bd)

Co-authored-by: Alexander Andreev <alexander.andreev@intel.com>
Co-authored-by: icfaust <icfaust@gmail.com>

ENH: Array API dispatching (#2096) (#2545)

* ENH: array api dispatching

added array-api-compat to test env

* Deselect some scikit-learn Array API tests

* deselect more tests

* deselect more tests

* disabled tests for

* fix the deselection comment

* disabled test for Ridge regression

* Disabled tests and added comment

* ENH: Array API dispatching

* Revert adding dpctl into Array PI conformance testing

added versioning for the get_nnamespace

* minor refactoring onedal _array_api

* add tests

* addressed memory usage tests

* Address some array api test fails

* linting

* addressed test_get_namespace

* adding test case for validate_data check with Array API inputs

* minor refactoring

* addressed test_patch_map_match fail

* Added docstrings for get_namespace

* docstrings for Array API tests

* updated minimal scikit-learn version for Array API dispatching

* updated minimal scikit-learn version for Array API dispatching in _device_offload.py _array_api.py

* fix test test_get_namespace_with_config_context

* refactor onedal/datatypes/_data_conversion.py

* correction for array api

* Update conftest.py

* introduce tags

* fix imports

* see if this works

* really lazy logic introduction

* introduce IntelEstimator

* missing change in knn

* recofigure logic

* strip out dpnp/dpctl special code, will come back to it later

* switchover

* Update __init__.py

* Update test_array_api.py

* merge main into PR

* try to fix changes:

* add test:

* attempt to get things running

* remove tsne failure

* try again

* add sklearn_check_version

* oops

* update

* another try

* try again

* Update test_common.py

* Update test_common.py

* Update test_common.py

* Update test_common.py

* Update test_common.py

* fixes

* attempt to re-introduce changes

* Update _device_offload.py

* switch get_tags

* merge main into branch

* remove leftover code

* Update base.py

* formatting

* correct bad sklearn recommendation

* remove irrelevant new tests

* fix mistake

* lint fix

* add fallback test

* make test fixes

* formatting

* rejigger the logic again

* Update test_config.py

* interim fix

* Update test_config.py

* attempt to get test_config test to skip

* fix test

* interim fixes

* fixes for IncrementalEmpiricalCovariance

* fixes for wrap_output_data

* solve issues related to older numpy

* Update test_run_to_run_stability.py

* Update __init__.py

* Update __init__.py

* Update test_run_to_run_stability.py

* Update base.py

* Update test_run_to_run_stability.py

* Update test_config.py

* Update test_patching.py

* Update test_patching.py

* Update test_patching.py

* Update k_means.py

* Update k_means.py

* Update k_means.py

* Update _array_api.py

* Update _array_api.py

* Update _device_offload.py

* Update _device_offload.py

* Update _device_offload.py

* Update _device_offload.py

* Update test_patching.py

* Update test_patching.py

* Update test_patching.py

* local verify doclinks

* fix doclink

* fixes

* remove print statement

* fix changes from scikit-learn/scikit-learn#29774

* Update sklearnex/_device_offload.py



* add type hints and fix docs for dispatch

* remove change from local testing

* be more explicit with a type

* python3.9 fix

* change language which was bothering me

* formatting

* This should fix it (I think)

* Apply suggestions from code review



* Update base.py

* Update __init__.py

* remove hack on config

* Apply suggestions from code review



* make suggested changes

* fix import in test_patching

* Update test_patching.py

* Update build-and-test-lnx.yml

* Update build-and-test-lnx.yml

* Update build-and-test-lnx.yml

* Apply suggestions from code review



* Apply suggestions from code review




* formatting and ABC change

* fix gpu test

* remove workaround

---------





(cherry picked from commit 43596ee)

Co-authored-by: Samir Nasibli <samir.nasibli@intel.com>
Co-authored-by: Faust, Ian <ian.faust@intel.com>
Co-authored-by: Ian Faust <icfaust@gmail.com>
Co-authored-by: david-cortes-intel <david.cortes@intel.com>
Co-authored-by: Andreas Huber <9201869+ahuber21@users.noreply.github.com>

MAINT: Remove separate daal4py docs (#2429)

* remove separate d4p docs

* remove/update more references

* update links