Playwright拦截接口并修改返回数据的N种方法

Playwright Mock 核心概念

什么是 API Mock?

API Mock 是一种测试技术,通过拦截前端请求并返回预设响应,实现:

  • 环境解耦:无需依赖真实后端服务
  • 数据控制:精准构造测试所需响应数据
  • 异常模拟:轻松复现网络错误、超时等场景

Playwright Mock 优势

  • 多语言支持:Java/Python/C#/Node.js 统一 API
  • 灵活匹配:支持 URL 正则、请求头/体内容匹配
  • 动态响应:可基于请求参数生成动态响应
  • 调试友好:内置请求日志记录功能

基础 Mock 示例

环境准备

<!-- Maven 依赖 -->
<dependency>
    <groupId>com.microsoft.playwright</groupId>
    <artifactId>playwright</artifactId>
    <version>1.32.0</version>
</dependency>

基础 Mock

import com.microsoft.playwright.*;

public class MockDemo {
    public static void main(String[] args) {
        try (Playwright playwright = Playwright.create()) {
            // 1. 创建浏览器实例
            Browser browser = playwright.chromium().launch();
            
            // 2. 创建上下文并启用 Mock
            BrowserContext context = browser.newContext(new Browser.NewContextOptions()
                .withRoute("https://api.example.com/data", route -> 
                    route.mock()
                        .withStatus(200)
                        .withBody("{\"code\":200,\"data\":[1,2,3]}")
                )
            );
            
            // 3. 执行测试
            Page page = context.newPage();
            page.navigate("https://your-frontend-app.com");
            System.out.println(page.content());
        }
    }
}

代码解析:

  1. withRoute() 定义需要拦截的 URL 模式
  2. mock() 方法链式配置响应参数:
    • withStatus() 设置 HTTP 状态码
    • withBody() 定义返回内容(支持 JSON/HTML/Text)
    • withHeaders() 可自定义响应头

复杂场景处理

动态响应生成

route.mock()
    .withStatus(200)
    .withBody(request -> {
        String requestId = request.url().split("id=")[1];
        return String.format("{\"id\":%s,\"name\":\"Mock User\"}", requestId);
    });

适用场景:需要根据请求参数动态生成响应内容(如用户ID查询)

多请求匹配策略

// 正则匹配 URL
context = browser.newContext(new Browser.NewContextOptions()
    .withRoute(/user\/\d+/, route -> 
        route.mock()
            .withBody("{\"status\":\"OK\"}")
    )
);

// 匹配请求头
context = browser.newContext(new Browser.NewContextOptions()
    .withRoute(route -> 
        route.matchHeader("Authorization", "Bearer mock-token")
    )
);

模拟网络异常

route.mock()
    .withStatus(500)
    .withBody("Server Error")
    .withDelay(2000); // 模拟 2 秒延迟

扩展用法

  • withAbort() 模拟连接中断
  • withTimeout() 设置响应超时

实战案例

测试目标

验证用户登录功能在不同响应下的表现:

  1. 正常登录(返回 200 + token)
  2. 密码错误(返回 401)
  3. 服务器异常(返回 500)

测试代码实现

public class LoginTest {
    private static final String API_URL = "https://api.example.com/login";
    
    @Test
    public void testLoginSuccess() {
        try (Playwright playwright = Playwright.create()) {
            BrowserContext context = createMockContext(200, "{\"token\":\"valid-token\"}");
            
            Page page = context.newPage();
            page.navigate("https://app.example.com/login");
            page.fill("#username", "testuser");
            page.fill("#password", "correctpass");
            page.click("button[type=submit]");
            
            assertThat(page.innerText(".welcome-message")).contains("Welcome");
        }
    }

    @Test
    public void testWrongPassword() {
        try (Playwright playwright = Playwright.create()) {
            BrowserContext context = createMockContext(401, "{\"error\":\"Invalid credentials\"}");
            
            Page page = context.newPage();
            page.navigate("https://app.example.com/login");
            // ... 执行登录操作
            assertThat(page.innerText(".error-message")).contains("Wrong password");
        }
    }

    private BrowserContext createMockContext(int status, String body) {
        return playwright.chromium().launch().newContext(new Browser.NewContextOptions()
            .withRoute(API_URL, route -> 
                route.mock()
                    .withStatus(status)
                    .withBody(body)
            )
        );
    }
}

测试框架集成

  • JUnit 5:使用 @BeforeEach 初始化 Playwright 实例
  • TestNG:通过 @BeforeMethod 配置上下文
  • 断言库:推荐结合 AssertJ 进行流式断言
package main import ( "errors" "fmt" "image" "image/color" "log" "net/http" "os" "path/filepath" "runtime" "strconv" "strings" "time" "fyne.io/fyne/v2" "fyne.io/fyne/v2/app" "fyne.io/fyne/v2/canvas" "fyne.io/fyne/v2/container" "fyne.io/fyne/v2/data/binding" "fyne.io/fyne/v2/dialog" "fyne.io/fyne/v2/layout" "fyne.io/fyne/v2/widget" "github.com/playwright-community/playwright-go" "main.go/dataModel/CookieModel" "main.go/dataModel/ShopModel" "main.go/dataModel/SkuModel" "main.go/dataModel/UserModel" "main.go/res" "main.go/tuuz/database" ) // PlaywrightService 管理Playwright实例 type PlaywrightService struct { Browser playwright.Browser Context playwright.BrowserContext Page playwright.Page } // 新增分页状态结构体 type PaginationState struct { CurrentPage int PageSize int TotalPages int TotalProducts int Products []SkuModel.DataItem } // 全局状态 type AppState struct { Window fyne.Window CurrentUser UserModel.UserInfo Shops []ShopModel.Account ProductTabs *container.AppTabs StatusBar *widget.Label ShopListBinding binding.UntypedList LoginForm *widget.Form LeftPanel *fyne.Container FilterFilePath string FilterKeywords []string ShopListPanel *fyne.Container FilterPanel *fyne.Container KeywordCount *widget.Label TabShopMap map[string]ShopModel.Account SplitContainer *container.Split TopPanel *fyne.Container ContentPanel *fyne.Container NeedsRefresh bool LastRefreshTime time.Time PaginationStates map[string]*PaginationState Playwright *PlaywrightService // Playwright服务 UrlEntry *widget.Entry // URL输入框 } // 添加状态检查快捷键 func addStateDebugShortcut(window fyne.Window, appState *AppState) { window.Canvas().SetOnTypedKey(func(ev *fyne.KeyEvent) { if ev.Name == fyne.KeyF5 { refreshLeftPanel(appState) appState.StatusBar.SetText("手动刷新UI") } else if ev.Name == fyne.KeyS { fmt.Println("===== 应用状态快照 =====") fmt.Printf("当前用户: %s\n", appState.CurrentUser.LoginName) fmt.Printf("店铺数量: %d\n", len(appState.Shops)) fmt.Printf("最后刷新时间: %s\n", appState.LastRefreshTime.Format("15:04:05.000")) fmt.Println("=======================") } }) } // 初始化Playwright服务 func initPlaywrightService() (*PlaywrightService, error) { pw, err := playwright.Run() if err != nil { return nil, fmt.Errorf("启动Playwright失败: %w", err) } browser, err := pw.Chromium.Launch(playwright.BrowserTypeLaunchOptions{ Headless: playwright.Bool(false), }) if err != nil { return nil, fmt.Errorf("启动浏览器失败: %w", err) } context, err := browser.NewContext() if err != nil { return nil, fmt.Errorf("创建上下文失败: %w", err) } page, err := context.NewPage() if err != nil { return nil, fmt.Errorf("创建页面失败: %w", err) } return &PlaywrightService{ Browser: browser, Context: context, Page: page, }, nil } func main() { os.Setenv("PLAYWRIGHT_BROWSERS_PATH", "./browsers") database.Init() UserModel.UserInit() ShopModel.ShopInit() CookieModel.CreateCookieInfoTable() SkuModel.ProductInit() myApp := app.New() myWindow := myApp.NewWindow("店铺管理工具") myWindow.Resize(fyne.NewSize(1200, 800)) // 初始化Playwright服务 pwService, err := initPlaywrightService() if err != nil { log.Fatalf("初始化Playwright失败: %v", err) } defer func() { if err := pwService.Browser.Close(); err != nil { log.Printf("关闭浏览器失败: %v", err) } if err := playwright.Stop(); err != nil { log.Printf("停止Playwright失败: %v", err) } }() // 初始化应用状态 appState := &AppState{ FilterFilePath: getDefaultFilterPath(), TabShopMap: make(map[string]ShopModel.Account), LastRefreshTime: time.Now(), PaginationStates: make(map[string]*PaginationState), Playwright: pwService, // 注入Playwright服务 } // 注册调试快捷键 addStateDebugShortcut(myWindow, appState) // 启动状态监听器 startStateListener(appState) // 尝试加载默认过滤文件 go loadFilterFile(appState) // 创建状态栏 appState.StatusBar = widget.NewLabel("就绪") // 创建URL访问控件 appState.UrlEntry = widget.NewEntry() appState.UrlEntry.SetPlaceHolder("输入URL") visitButton := widget.NewButton("访问", func() { url := appState.UrlEntry.Text if url == "" { appState.StatusBar.SetText("请输入URL") return } appState.StatusBar.SetText(fmt.Sprintf("正在访问: %s...", url)) go func() { // 访问URL response, err := visitUrlWithPlaywright(appState, url) if err != nil { fyne.DoAndWait(func() { appState.StatusBar.SetText(fmt.Sprintf("访问失败: %v", err)) }) return } fyne.DoAndWait(func() { appState.StatusBar.SetText(fmt.Sprintf("访问完成! 状态码: %d", response.status)) }) }() }) // 创建底部控制栏 bottomControlBar := container.NewBorder( nil, nil, nil, visitButton, appState.UrlEntry, ) // 创建底部区域(状态栏 + URL控件) bottomArea := container.NewVBox( bottomControlBar, widget.NewSeparator(), container.NewHBox(layout.NewSpacer(), appState.StatusBar), ) // 创建主布局 mainContent := createMainUI(myWindow, appState) // 设置整体布局 content := container.NewBorder( nil, // 顶部 bottomArea, // 底部(包含URL控件和状态栏) nil, // 左侧 nil, // 右侧 mainContent, ) myWindow.SetContent(content) // 启动时尝试自动登录 go tryAutoLogin(appState) myWindow.ShowAndRun() } // 使用Playwright访问URL拦截响应 func visitUrlWithPlaywright(appState *AppState, url string) (*playwright.Response, error) { // 设置响应拦截器 appState.Playwright.Page.OnResponse(func(response playwright.Response) { log.Printf("响应: %s - %d", response.URL(), response.Status()) }) // 导航到URL response, err := appState.Playwright.Page.Goto(url) if err != nil { return nil, fmt.Errorf("导航失败: %w", err) } // 等待页面加载完成 if err := appState.Playwright.Page.WaitForLoadState(playwright.PageWaitForLoadStateOptions{ State: playwright.LoadStateNetworkidle, }); err != nil { return nil, fmt.Errorf("等待页面加载失败: %w", err) } return response, nil } // 新增状态监听器 - 定期检查状态变化 func startStateListener(appState *AppState) { go func() { for { time.Sleep(100 * time.Millisecond) // 每100ms检查一次 if appState.NeedsRefresh { fyne.DoAndWait(func() { refreshLeftPanel(appState) appState.NeedsRefresh = false }) } } }() } // 获取默认过滤文件路径 func getDefaultFilterPath() string { if runtime.GOOS == "windows" { return filepath.Join("filter.txt") } return filepath.Join(os.Getenv("HOME"), "filter.txt") } // 修改 refreshAllProductTabs 函数 func refreshAllProductTabs(appState *AppState) { if appState.ProductTabs == nil || len(appState.ProductTabs.Items) == 0 { return } // 遍历所有标签页刷新 for _, tab := range appState.ProductTabs.Items { // 通过标签页标题获取店铺 shop, exists := appState.TabShopMap[tab.Text] if !exists { continue } // 重新加载商品 go func(shop ShopModel.Account) { products, err := loadProductsForShop(shop, appState) if err != nil { fyne.DoAndWait(func() { appState.StatusBar.SetText(fmt.Sprintf("刷新 %s 商品失败: %s", shop.AccountName, err.Error())) }) return } fyne.DoAndWait(func() { // 更新标签页内容 tab.Content = container.NewMax(createProductList(products)) appState.ProductTabs.Refresh() appState.StatusBar.SetText(fmt.Sprintf("已刷新 %s 的商品", shop.AccountName)) }) }(shop) } } // 加载过滤文件 func loadFilterFile(appState *AppState) { if appState.FilterFilePath == "" { log.Printf("加载本地过滤文件失败: %s", appState.FilterFilePath) return } if _, err := os.Stat(appState.FilterFilePath); os.IsNotExist(err) { err := os.WriteFile(appState.FilterFilePath, []byte{}, 0644) if err != nil { log.Printf("创建过滤文件失败: %v", err) } return } content, err := os.ReadFile(appState.FilterFilePath) if err != nil { log.Printf("读取过滤文件失败: %v", err) return } lines := strings.Split(string(content), "\n") appState.FilterKeywords = []string{} for _, line := range lines { trimmed := strings.TrimSpace(line) if trimmed != "" { appState.FilterKeywords = append(appState.FilterKeywords, trimmed) } } fyne.DoAndWait(func() { appState.StatusBar.SetText(fmt.Sprintf("已加载 %d 个过滤关键字", len(appState.FilterKeywords))) // 更新关键字数量标签 if appState.KeywordCount != nil { // 修正为 KeywordCount appState.KeywordCount.SetText(fmt.Sprintf("关键字数量: %d", len(appState.FilterKeywords))) } // 刷新所有已打开的商品标签页 refreshAllProductTabs(appState) }) } // 修改 createMainUI 函数 - 保存分割布局引用 func createMainUI(window fyne.Window, appState *AppState) fyne.CanvasObject { appState.Window = window // 创建整个左侧面板 leftPanel := createLeftPanel(window, appState) appState.LeftPanel = leftPanel // 右侧面板 appState.ProductTabs = container.NewAppTabs() appState.ProductTabs.SetTabLocation(container.TabLocationTop) rightPanel := container.NewBorder( widget.NewLabelWithStyle("商品信息", fyne.TextAlignCenter, fyne.TextStyle{Bold: true}), nil, nil, nil, container.NewMax(appState.ProductTabs), ) // 使用HSplit布局 - 保存引用 split := container.NewHSplit(leftPanel, rightPanel) split.SetOffset(0.25) appState.SplitContainer = split // 保存分割布局引用 return split } // 修改createFilterPanel函数 - 返回容器保存引用 func createFilterPanel(appState *AppState) *fyne.Container { // 创建文件路径标签 pathLabel := widget.NewLabel("过滤文件: " + appState.FilterFilePath) pathLabel.Wrapping = fyne.TextWrapWord // 创建选择文件按钮 selectButton := widget.NewButton("选择过滤文件", func() { dialog.ShowFileOpen(func(reader fyne.URIReadCloser, err error) { if err != nil { dialog.ShowError(err, appState.Window) return } if reader == nil { return // 用户取消 } // 更新文件路径 appState.FilterFilePath = reader.URI().Path() pathLabel.SetText("过滤文件: " + appState.FilterFilePath) // 加载过滤文件 go func() { loadFilterFile(appState) // 刷新所有已打开的商品标签页 refreshAllProductTabs(appState) }() }, appState.Window) }) // 创建刷新按钮 refreshButton := widget.NewButton("刷新过滤", func() { if appState.FilterFilePath != "" { appState.StatusBar.SetText("刷新过滤关键字...") go func() { loadFilterFile(appState) // 刷新所有已打开的商品标签页 refreshAllProductTabs(appState) }() } else { appState.StatusBar.SetText("请先选择过滤文件") } }) // 创建"增加商品"按钮 addProductsButton := widget.NewButton("增加商品", func() { if appState.ProductTabs.Selected() == nil { appState.StatusBar.SetText("请先选择一个店铺标签页") return } shopName := appState.ProductTabs.Selected().Text appState.StatusBar.SetText(fmt.Sprintf("为 %s 增加1000条商品...", shopName)) go func() { // 生成1000条模拟商品 newProducts := make([]SkuModel.DataItem, 1000) for i := 0; i < 1000; i++ { newProducts[i] = SkuModel.DataItem{ ProductID: fmt.Sprintf("ADD%04d", i+1), Name: fmt.Sprintf("%s - 新增商品%d", shopName, i+1), MarketPrice: (i + 1000) * 1000, // 从1000开始 DiscountPrice: (i + 1000) * 800, // 折扣价 } } fyne.DoAndWait(func() { // 获取该店铺的TabState tabState, exists := appState.PaginationStates[shopName] if !exists { // 如果不存在,则创建一个新的TabState tabState = &PaginationState{ PageSize: 10, CurrentPage: 1, } appState.PaginationStates[shopName] = tabState } // 添加到现有商品列表 tabState.Products = append(tabState.Products, newProducts...) // 刷新当前标签页 refreshCurrentProductTab(appState, shopName, tabState.Products) appState.StatusBar.SetText(fmt.Sprintf("已为 %s 增加1000条商品,总数: %d", shopName, len(tabState.Products))) }) }() }) // 修改按钮容器,添加新按钮 buttonContainer := container.NewHBox( selectButton, refreshButton, addProductsButton, // 新增按钮 ) // 创建关键字计数标签 - 保存引用 keywordCount := widget.NewLabel(fmt.Sprintf("关键字数量: %d", len(appState.FilterKeywords))) keywordCount.TextStyle = fyne.TextStyle{Bold: true} appState.KeywordCount = keywordCount // 创建面板容器 panel := container.NewVBox( widget.NewSeparator(), widget.NewLabelWithStyle("商品过滤", fyne.TextAlignCenter, fyne.TextStyle{Bold: true}), pathLabel, keywordCount, buttonContainer, ) return panel } // 修改 createLoggedInPanel 函数 - 确保注销时直接刷新 func createLoggedInPanel(appState *AppState) fyne.CanvasObject { return container.NewVBox( widget.NewLabelWithStyle("登录状态", fyne.TextAlignCenter, fyne.TextStyle{Bold: true}), widget.NewSeparator(), container.NewHBox( widget.NewLabel("用户:"), widget.NewLabel(appState.CurrentUser.LoginName), ), container.NewHBox( widget.NewLabel("店铺数量:"), widget.NewLabel(fmt.Sprintf("%d", len(appState.Shops))), ), widget.NewSeparator(), container.NewCenter( widget.NewButton("注销", func() { // 重置状态 appState.CurrentUser = UserModel.UserInfo{} appState.Shops = nil appState.ProductTabs.Items = nil appState.ProductTabs.Refresh() appState.TabShopMap = make(map[string]ShopModel.Account) // 直接调用刷新函数 refreshLeftPanel(appState) appState.StatusBar.SetText("已注销") }), ), ) } // 重构创建顶部面板函数 - 确保状态正确反映 func createTopPanel(appState *AppState) *fyne.Container { // 添加调试日志 fmt.Printf("创建顶部面板: 登录状态=%t, 用户名=%s\n", appState.CurrentUser.LoginName != "", appState.CurrentUser.LoginName) var content fyne.CanvasObject if appState.CurrentUser.LoginName != "" { content = createLoggedInPanel(appState) } else { content = createLoginForm(appState) } return container.NewMax(content) } // 重构 createContentPanel 函数 - 添加详细日志 func createContentPanel(appState *AppState) *fyne.Container { // 添加详细调试日志 fmt.Printf("创建内容面板: 登录状态=%t, 用户名=%s, 店铺数量=%d\n", appState.CurrentUser.LoginName != "", appState.CurrentUser.LoginName, len(appState.Shops)) if appState.CurrentUser.LoginName != "" { if len(appState.Shops) > 0 { return createShopListPanel(appState) } return container.NewCenter( widget.NewLabel("没有可用的店铺"), ) } return container.NewCenter( widget.NewLabel("请先登录查看店铺列表"), ) } // 重构刷新函数 - 确保完全重建UI func refreshLeftPanel(appState *AppState) { if appState.SplitContainer == nil { return } // 添加详细调试信息 fmt.Printf("刷新左侧面板 - 时间: %s, 用户: %s, 店铺数量: %d\n", time.Now().Format("15:04:05.000"), appState.CurrentUser.LoginName, len(appState.Shops)) // 创建新的左侧面板 newLeftPanel := createLeftPanel(appState.Window, appState) // 添加调试背景色(登录状态不同颜色不同) var debugColor color.Color if appState.CurrentUser.LoginName != "" { debugColor = color.NRGBA{R: 0, G: 100, B: 0, A: 30} // 登录状态绿色半透明 } else { debugColor = color.NRGBA{R: 100, G: 0, B: 0, A: 30} // 未登录状态红色半透明 } debugPanel := container.NewMax( canvas.NewRectangle(debugColor), newLeftPanel, ) // 替换分割布局中的左侧面板 appState.SplitContainer.Leading = debugPanel appState.LeftPanel = debugPanel // 刷新分割布局 appState.SplitContainer.Refresh() // 强制重绘整个窗口 appState.Window.Content().Refresh() appState.LastRefreshTime = time.Now() } // 重构 createLeftPanel 函数 - 确保使用正确的状态 func createLeftPanel(window fyne.Window, appState *AppState) *fyne.Container { // 创建顶部面板(用户状态/登录表单) topPanel := createTopPanel(appState) // 创建内容面板(店铺列表或提示) contentPanel := createContentPanel(appState) // 创建过滤面板 filterPanel := createFilterPanel(appState) // 使用Border布局 return container.NewBorder( topPanel, // 顶部 filterPanel, // 底部 nil, nil, // 左右 contentPanel, // 中间内容 ) } // 修改登录按钮回调 - 确保状态正确更新 func createLoginForm(appState *AppState) fyne.CanvasObject { usernameEntry := widget.NewEntry() passwordEntry := widget.NewPasswordEntry() usernameEntry.PlaceHolder = "输入邮箱地址" passwordEntry.PlaceHolder = "输入密码" // 登录按钮回调 loginButton := widget.NewButton("登录", func() { appState.StatusBar.SetText("登录中...") go func() { // 模拟网络延迟 time.Sleep(500 * time.Millisecond) // 获取店铺信息 shops := ShopModel.Api_select_struct(nil) fyne.DoAndWait(func() { if len(shops) == 0 { appState.StatusBar.SetText("获取店铺信息为空") return } // 更新应用状态 appState.Shops = shops appState.CurrentUser, _ = UserModel.Api_find_by_username(usernameEntry.Text) // 更新店铺列表绑定 updateShopListBinding(appState) // 新增:更新绑定数据 // 添加状态更新日志 fmt.Printf("登录成功 - 用户: %s, 店铺数量: %d\n", appState.CurrentUser.LoginName, len(appState.Shops)) if appState.CurrentUser.LoginName == "" { appState.CurrentUser.LoginName = "1" } appState.StatusBar.SetText(fmt.Sprintf("登录成功! 共 %d 个店铺", len(shops))) // 直接刷新UI refreshLeftPanel(appState) }) }() }) form := widget.NewForm( widget.NewFormItem("邮箱:", usernameEntry), widget.NewFormItem("密码:", passwordEntry), ) appState.LoginForm = form return container.NewVBox( widget.NewLabelWithStyle("登录面板", fyne.TextAlignCenter, fyne.TextStyle{Bold: true}), form, container.NewCenter(loginButton), ) } // 修改自动登录函数 - 添加详细日志 func tryAutoLogin(appState *AppState) { // 获取所有用户 users := UserModel.Api_select_struct(nil) if len(users) == 0 { fyne.DoAndWait(func() { appState.StatusBar.SetText("获取已经存在的账号为空") }) return } // 尝试使用第一个用户自动登录 user := users[0] fyne.DoAndWait(func() { appState.StatusBar.SetText(fmt.Sprintf("尝试自动登录: %s...", user.LoginName)) }) // 获取用户名输入框 if appState.LoginForm == nil || len(appState.LoginForm.Items) < 2 { fyne.DoAndWait(func() { appState.StatusBar.SetText("自动登录失败: 登录表单尚未初始化") }) return } usernameItem := appState.LoginForm.Items[0] usernameEntry, ok := usernameItem.Widget.(*widget.Entry) if !ok { fyne.DoAndWait(func() { appState.StatusBar.SetText("自动登录失败: 用户名控件类型错误") }) return } passwordItem := appState.LoginForm.Items[1] passwordEntry, ok := passwordItem.Widget.(*widget.Entry) if !ok { fyne.DoAndWait(func() { appState.StatusBar.SetText("自动登录失败: 密码控件类型错误") }) return } // 触发登录 fyne.DoAndWait(func() { usernameEntry.SetText(user.LoginName) passwordEntry.SetText(user.LoginPass) appState.StatusBar.SetText("正在自动登录...") // 更新应用状态 appState.CurrentUser = user appState.Shops = ShopModel.Api_select_struct(nil) // 更新店铺列表绑定 updateShopListBinding(appState) // 新增 // 添加自动登录日志 fmt.Printf("自动登录成功 - 用户: %s, 店铺数量: %d\n", appState.CurrentUser.LoginName, len(appState.Shops)) // 直接刷新UI refreshLeftPanel(appState) }) } // 修改后的异步加载店铺头像函数 func loadShopAvatar(img *canvas.Image, url string) { if url == "" { // 使用默认头像 fyne.DoAndWait(func() { img.Resource = fyne.Theme.Icon(fyne.CurrentApp().Settings().Theme(), "account") img.Refresh() }) return } // 创建HTTP客户端(可设置超时) client := &http.Client{ Timeout: 10 * time.Second, } resp, err := client.Get(url) if err != nil { log.Printf("加载头像失败: %v", err) return } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { log.Printf("头像请求失败: %s", resp.Status) return } // 解码图片 imgData, _, err := image.Decode(resp.Body) if err != nil { log.Printf("解码头像失败: %v", err) return } // 在主线程更新UI fyne.DoAndWait(func() { img.Image = imgData img.Refresh() }) } // 修改后的 createShopListPanel 函数 func createShopListPanel(appState *AppState) *fyne.Container { // 创建绑定数据 if appState.ShopListBinding == nil { appState.ShopListBinding = binding.NewUntypedList() } else { // 确保绑定数据是最新的 updateShopListBinding(appState) } // 创建列表控件 list := widget.NewListWithData( appState.ShopListBinding, func() fyne.CanvasObject { avatar := canvas.NewImageFromResource(nil) avatar.SetMinSize(fyne.NewSize(40, 40)) avatar.FillMode = canvas.ImageFillContain nameLabel := widget.NewLabel("") statusIcon := widget.NewIcon(nil) return container.NewHBox( avatar, container.NewVBox(nameLabel), layout.NewSpacer(), statusIcon, ) }, func(item binding.DataItem, obj fyne.CanvasObject) { hbox, ok := obj.(*fyne.Container) if !ok || len(hbox.Objects) < 4 { return } avatar, _ := hbox.Objects[0].(*canvas.Image) nameContainer, _ := hbox.Objects[1].(*fyne.Container) nameLabel, _ := nameContainer.Objects[0].(*widget.Label) statusIcon, _ := hbox.Objects[3].(*widget.Icon) val, err := item.(binding.Untyped).Get() if err != nil { return } shop, ok := val.(ShopModel.Account) if !ok { return } nameLabel.SetText(shop.AccountName) if shop.CanLogin { statusIcon.SetResource(res.ResShuffleSvg) } else { statusIcon.SetResource(fyne.Theme.Icon(fyne.CurrentApp().Settings().Theme(), "error")) } go loadShopAvatar(avatar, shop.AccountAvatar) }, ) list.OnSelected = func(id widget.ListItemID) { if id < 0 || id >= len(appState.Shops) { return } shop := appState.Shops[id] appState.StatusBar.SetText(fmt.Sprintf("加载 %s 的商品...", shop.AccountName)) go func() { products, err := loadProductsForShop(shop, appState) if err != nil { fyne.DoAndWait(func() { appState.StatusBar.SetText("加载商品失败: " + err.Error()) }) return } fyne.DoAndWait(func() { appState.StatusBar.SetText(fmt.Sprintf("已加载 %d 个商品", len(products))) addOrUpdateProductTab(appState, shop, products) }) }() } // 创建滚动容器 - 设置最小高度确保可滚动 scrollContainer := container.NewScroll(list) scrollContainer.SetMinSize(fyne.NewSize(280, 200)) // 最小高度200确保可滚动 // 使用Max容器确保填充空间 return container.NewMax( container.NewBorder( widget.NewLabel("店铺列表"), nil, nil, nil, scrollContainer, ), ) } // 更新店铺列表绑定数据 func updateShopListBinding(appState *AppState) { if appState.ShopListBinding == nil { appState.ShopListBinding = binding.NewUntypedList() } values := make([]interface{}, len(appState.Shops)) for i, shop := range appState.Shops { values[i] = shop } appState.ShopListBinding.Set(values) } // 应用商品过滤 func applyProductFilter(products []SkuModel.DataItem, keywords []string) []SkuModel.DataItem { if len(keywords) == 0 { return products // 没有关键字,返回所有商品 } filtered := []SkuModel.DataItem{} for _, product := range products { exclude := false for _, keyword := range keywords { if strings.Contains(strings.ToLower(product.Name), strings.ToLower(keyword)) { exclude = true break } } if !exclude { filtered = append(filtered, product) } } return filtered } // 修改 loadProductsForShop 函数,生成更多模拟数据 func loadProductsForShop(shop ShopModel.Account, appState *AppState) ([]SkuModel.DataItem, error) { // 模拟API调用获取商品数据 time.Sleep(500 * time.Millisecond) // 模拟网络延迟 // 生成100条模拟商品数据 products := make([]SkuModel.DataItem, 100) for i := 0; i < 100; i++ { products[i] = SkuModel.DataItem{ ProductID: fmt.Sprintf("SKU%04d", i+1), Name: fmt.Sprintf("%s - 商品%d", shop.AccountName, i+1), MarketPrice: i * 1000, DiscountPrice: i * 1000, } } // 应用过滤 filteredProducts := applyProductFilter(products, appState.FilterKeywords) return filteredProducts, nil } // 修改 addOrUpdateProductTab 函数,添加分页支持 func addOrUpdateProductTab(appState *AppState, shop ShopModel.Account, products []SkuModel.DataItem) { tabTitle := shop.AccountName // 获取或创建分页状态 pagination, exists := appState.PaginationStates[tabTitle] if !exists { // 初始化分页状态 pagination = &PaginationState{ PageSize: 10, CurrentPage: 1, TotalProducts: len(products), } appState.PaginationStates[tabTitle] = pagination } else { // 更新商品总数 pagination.TotalProducts = len(products) } // 计算总页数 pagination.TotalPages = (pagination.TotalProducts + pagination.PageSize - 1) / pagination.PageSize if pagination.TotalPages == 0 { pagination.TotalPages = 1 } // 获取当前页数据 currentPageProducts := getCurrentPageProducts(pagination, products) // 检查是否已存在该TAB for _, tab := range appState.ProductTabs.Items { if tab.Text == tabTitle { // 修改调用,传入店铺名称 tab.Content = createProductListWithPagination(appState, currentPageProducts, tabTitle, products) // 更新映射 appState.TabShopMap[tabTitle] = shop appState.ProductTabs.Refresh() return } } // 创建新TAB newTab := container.NewTabItem( tabTitle, createProductListWithPagination(appState, currentPageProducts, tabTitle, products), ) // 添加到映射 appState.TabShopMap[tabTitle] = shop appState.ProductTabs.Append(newTab) appState.ProductTabs.Select(newTab) } // 修改 getCurrentPageProducts 函数 func getCurrentPageProducts(pagination *PaginationState, products []SkuModel.DataItem) []SkuModel.DataItem { start := (pagination.CurrentPage - 1) * pagination.PageSize if start >= len(products) { start = 0 } end := start + pagination.PageSize if end > len(products) { end = len(products) } return products[start:end] } // 修改 createProductListWithPagination 函数 func createProductListWithPagination(appState *AppState, currentPageProducts []SkuModel.DataItem, shopName string, allProducts []SkuModel.DataItem) fyne.CanvasObject { // 创建表格 table := createProductTable(currentPageProducts) // 创建分页控件 - 传入店铺名称 pagination := createPaginationControls(appState, shopName, allProducts) // 创建布局:表格在上,分页控件在下 return container.NewBorder(nil, pagination, nil, nil, table) } // 创建商品表格 func createProductTable(products []SkuModel.DataItem) fyne.CanvasObject { // 创建表格 table := widget.NewTable( func() (int, int) { return len(products) + 1, 4 // 行数=商品数+表头,列数=4 }, func() fyne.CanvasObject { return widget.NewLabel("模板文本") }, func(id widget.TableCellID, cell fyne.CanvasObject) { label := cell.(*widget.Label) if id.Row == 0 { // 表头 switch id.Col { case 0: label.SetText("商品ID") case 1: label.SetText("商品名称") case 2: label.SetText("价格") case 3: label.SetText("库存") } label.TextStyle.Bold = true return } // 数据行 product := products[id.Row-1] switch id.Col { case 0: label.SetText(product.ProductID) case 1: label.SetText(product.Name) case 2: label.SetText(fmt.Sprintf("¥%.2f", float64(product.MarketPrice)/100)) case 3: label.SetText(fmt.Sprintf("%d", product.DiscountPrice)) } }, ) // 设置列宽 table.SetColumnWidth(0, 100) table.SetColumnWidth(1, 300) table.SetColumnWidth(2, 100) table.SetColumnWidth(3, 100) // 创建滚动容器 scrollContainer := container.NewScroll(table) scrollContainer.SetMinSize(fyne.NewSize(600, 400)) return scrollContainer } // 修改 createPaginationControls 函数 func createPaginationControls(appState *AppState, shopName string, allProducts []SkuModel.DataItem) *fyne.Container { // 获取该店铺的分页状态 pagination, exists := appState.PaginationStates[shopName] if !exists { // 如果不存在,创建默认状态 pagination = &PaginationState{ PageSize: 10, CurrentPage: 1, TotalProducts: len(allProducts), TotalPages: (len(allProducts) + 9) / 10, } appState.PaginationStates[shopName] = pagination } // 使用闭包捕获当前店铺名称 refreshForShop := func() { refreshCurrentProductTab(appState, shopName, allProducts) } // 上一页按钮 prevBtn := widget.NewButton("上一页", func() { if pagination.CurrentPage > 1 { pagination.CurrentPage-- refreshForShop() } }) // 页码信息 pageInfo := widget.NewLabel(fmt.Sprintf("第 %d 页/共 %d 页", pagination.CurrentPage, pagination.TotalPages)) // 下一页按钮 nextBtn := widget.NewButton("下一页", func() { if pagination.CurrentPage < pagination.TotalPages { pagination.CurrentPage++ refreshForShop() } }) // 跳转输入框 jumpEntry := widget.NewEntry() jumpEntry.SetPlaceHolder("页码") jumpEntry.Validator = func(s string) error { _, err := strconv.Atoi(s) if err != nil { return errors.New("请输入数字") } return nil } jumpBtn := widget.NewButton("跳转", func() { page, err := strconv.Atoi(jumpEntry.Text) if err == nil && page >= 1 && page <= pagination.TotalPages { pagination.CurrentPage = page refreshForShop() } }) // 页面大小选择器 pageSizeSelect := widget.NewSelect([]string{"5", "10", "20", "50"}, nil) pageSizeSelect.SetSelected(fmt.Sprintf("%d", pagination.PageSize)) pageSizeSelect.OnChanged = func(value string) { size, _ := strconv.Atoi(value) pagination.PageSize = size pagination.CurrentPage = 1 // 重新计算总页数 pagination.TotalPages = (len(allProducts) + pagination.PageSize - 1) / pagination.PageSize if pagination.TotalPages == 0 { pagination.TotalPages = 1 } refreshForShop() } pageSizeLabel := widget.NewLabel("每页:") // 布局 return container.NewHBox( prevBtn, pageSizeLabel, pageSizeSelect, pageInfo, nextBtn, jumpEntry, jumpBtn, ) } // 修改 refreshCurrentProductTab 函数 func refreshCurrentProductTab(appState *AppState, shopName string, allProducts []SkuModel.DataItem) { // 获取当前选中的标签页 currentTab := appState.ProductTabs.Selected() if currentTab == nil { return } // 获取该店铺的分页状态 pagination, exists := appState.PaginationStates[shopName] if !exists { // 如果不存在,创建默认状态 pagination = &PaginationState{ PageSize: 10, CurrentPage: 1, TotalProducts: len(allProducts), } appState.PaginationStates[shopName] = pagination } // 更新商品总数 pagination.TotalProducts = len(allProducts) // 计算总页数 pagination.TotalPages = (pagination.TotalProducts + pagination.PageSize - 1) / pagination.PageSize if pagination.TotalPages == 0 { pagination.TotalPages = 1 } // 获取当前页数据 currentPageProducts := getCurrentPageProducts(pagination, allProducts) // 更新标签页内容 currentTab.Content = createProductListWithPagination(appState, currentPageProducts, shopName, allProducts) appState.ProductTabs.Refresh() } // 创建商品列表 - 修复表格填充问题 func createProductList(products []SkuModel.DataItem) fyne.CanvasObject { // 创建表格 table := widget.NewTable( func() (int, int) { return len(products) + 1, 4 // 行数=商品数+表头,列数=4 }, func() fyne.CanvasObject { return widget.NewLabel("模板文本") }, func(id widget.TableCellID, cell fyne.CanvasObject) { label := cell.(*widget.Label) if id.Row == 0 { // 表头 switch id.Col { case 0: label.SetText("商品ID") case 1: label.SetText("商品名称") case 2: label.SetText("价格") case 3: label.SetText("库存") } label.TextStyle.Bold = true return } // 数据行 product := products[id.Row-1] switch id.Col { case 0: label.SetText(product.ProductID) case 1: label.SetText(product.Name) case 2: label.SetText(fmt.Sprintf("¥%.2f", float64(product.MarketPrice)/100)) case 3: label.SetText(fmt.Sprintf("%d", product.DiscountPrice)) } }, ) // 设置列宽 table.SetColumnWidth(0, 100) table.SetColumnWidth(1, 300) table.SetColumnWidth(2, 100) table.SetColumnWidth(3, 100) // 创建滚动容器 scrollContainer := container.NewScroll(table) scrollContainer.SetMinSize(fyne.NewSize(600, 400)) // 返回可滚动的表格容器 return scrollContainer } 这是修改后的代码,IDE提示3个编译错误,1:undefined: playwright.Stop,2:response.status undefined (type *playwright.Response is pointer to interface, not interface) 3:cannot use response (variable of interface type playwright.Response) as *playwright.Response value in return statement: playwright.Response does not implement *playwright.Response (type *playwright.Response is pointer to interface, not interface)
07-26
<think>嗯,用户想修改现有的Python脚本,从京东的商品页面抓取差评评论,还要实现翻页功能。首先,我需要考虑京东页面的结构,可能用的是动态加载,所以得用能处理JavaScript的库,比如Playwright的ChromiumPage。然后,差评通常是低分评论,京东的评分系统一般是1星或2星,可能需要找到对应的筛选参数或者元素。 接下来,URL分析。用户提供的链接是'https://item.jd.com/100095516785.html#comment',但实际评论数据可能通过API接口返回,而不是直接嵌入在HTML里。可能需要用浏览器的开发者工具抓取网络请求,找到真正的数据接口。比如,可能有个GET请求,参数包括productId、page页码、scoreType(差评分对应的值)。 翻页逻辑的话,需要确定总页数,可能从第一页的响应数据里获取总页数,然后循环请求每一页。另外,处理翻页时要注意防反爬机制,比如设置合理的延迟,或者使用代理。 然后,数据解析部分返回数据可能是JSON格式,需要用json模块解析,提取需要的字段,比如评论内容、评分、时间等。存储到CSV文件,需要用csv模块写入,确保处理中文编码问题,比如用utf-8-sig编码。 现有脚本可能需要调整的部分包括:修改URL为真正的API接口,添加page参数循环,处理差评的scoreType参数,解析JSON数据,写入CSV的步骤。同时,使用ChromiumPage模拟浏览器行为,可能需要等待元素加载,或者直接拦截API请求获取数据。 还需要考虑反爬措施,比如设置User-Agent,控制请求频率,避免被封IP。可能还需要处理登录状态,但京东的评论可能不需要登录就能获取。 测试的时候,要检查是否能正确获取多页数据,是否有重复或遗漏,以及数据是否正确存储到CSV中。可能遇到的坑包括参数错误导致获取不到差评,翻页逻辑错误,或者JSON结构变化导致解析失败。 总结下来,步骤应该是:1. 找到正确的API接口和参数;2. 设置差评过滤参数;3. 实现翻页循环;4. 解析JSON数据;5. 存储到CSV;6. 处理反爬问题。</think>以下是如何调整Python脚本实现京东商品差评采集与翻页功能的解决方案: ### 一、基础脚本调整 ```python from chromium_page import ChromiumPage import csv import json import time def jd_lowrate_crawler(product_id, max_pages): with ChromiumPage() as page: # 创建CSV文件 with open(f'jd_{product_id}_lowrate.csv', 'w', newline='', encoding='utf-8-sig') as f: writer = csv.writer(f) writer.writerow(['用户', '评分', '内容', '时间']) # 翻页逻辑 for page_num in range(1, max_pages+1): api_url = f'https://club.jd.com/comment/productPageComments.action?productId={product_id}&score=1&page={page_num}' page.goto(api_url) # 解析JSON数据 try: data = json.loads(page.content()) comments = data['comments'] for comment in comments: row = [ comment.get('nickname', ''), comment.get('score', ''), comment.get('content', '').strip(), comment.get('creationTime', '') ] writer.writerow(row) except Exception as e: print(f'第{page_num}页解析失败:', e) # 防反爬延迟 time.sleep(2) ``` ### 二、关键参数说明 1. **接口地址**:京东评论的真实接口为`club.jd.com/comment/productPageComments.action` 2. **核心参数**: - `productId`: 商品ID(从URL中提取) - `score=1`:表示差评(1=差评,2=中评,3=好评) - `page`:页码参数 ### 三、功能增强建议 1. **自动获取总页数**: ```python total_page = data['maxPage'] ``` 2. **代理配置**: ```python page = ChromiumPage(proxy='http://user:pass@ip:port') ``` 3. **异常重试机制**: ```python from retrying import retry @retry(stop_max_attempt_number=3, wait_fixed=2000) def safe_request(url): # 包装请求函数 ``` ### 四、执行示例 ```python if __name__ == "__main__": # 从原URL提取商品ID:100095516785 jd_lowrate_crawler(product_id='100095516785', max_pages=5) ``` ### 五、注意事项 1. 京东的反爬机制较严格,建议: - 保持`time.sleep(2)`以上间隔 - 使用动态User-Agent - 避免高频访问(建议每天采集不超过100页) 2. 数据存储建议添加去重机制,使用`comment['id']`作为唯一标识 3. 完整评论内容需要处理换行符: ```python content = comment['content'].replace('\n', ' ') ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值